Scottish officials say US memo giving grudging support to freeing Abdelbaset al-Megrahi undermines president's criticisms

Barack Obama is under growing pressure to release a letter that reveals the US grudgingly supported freeing the Lockerbie bomber on compassionate grounds.

The letter was sent to Scottish ministers by a senior diplomat at the US embassy in London last August, eight days before Abdelbaset al-Megrahi was released from prison because he was dying from inoperable prostate cancer.

Obama's administration has refused to allow publication of the letter, in which the US says allowing Megrahi to live at home in Scotland would be "far preferable" to sending him back to Libya under the prisoner transfer deal brokered by former prime minister Tony Blair in 2007.

Although Megrahi was allowed to go home to die in Tripoli, Scottish officials believe this undermines Obama's vigorous criticisms of the decision to free Megrahi earlier this month, when he said he was left "surprised, disappointed and angry" by the Libyan's release. David Cameron said he and Obama were in "violent agreement" that freeing Megrahi was a bad decision.

The American ambassador to the UK, Louis Susman, said the US was examining whether its correspondence on the issue could be released, but he refused to be drawn on the reported memo.

Senators in Washington are now pressing for the release of all correspondence between Obama's administration and the Scottish government in their investigation into allegations that BP directly influenced the decision to release Megrahi to help its $800m oil deal with Libya.

The allegations have been rejected by Alex Salmond, the first minister. He has offered to send dozens of official documents on the Megrahi affair to the US Senate's foreign relations committee after refusing to allow his justice minister, Kenny MacAskill, to appear at the committee's hearing on Lockerbie this Thursday.

Salmond insists the documents, released last year, prove that MacAskill released Megrahi solely on compassionate grounds and had completely ruled out using the prisoner transfer agreement brokered by Blair. But his aides have told the committee that Obama's government refused to allow some material to be published – including the US embassy correspondence.

Meanwhile, William Hague, the foreign secretary, came to Salmond's aid. In a seven-page letter to the committee he confirmed that BP had heavily lobbied the UK government about the prisoner transfer agreement with Libya.

Hague said BP met the former Labour government five times in October and November 2007 over its concerns that disputes about the treaty might damage its oil exploration contracts with Libya. But Hague said: "This was a perfectly normal and legitimate practice for a British company … There is no evidence that corroborates in any way the allegation of BP's involvement in the Scottish executive's entirely separate decision to release him on compassionate grounds."

The existence and content of the US embassy note was first disclosed by the Guardian last August, at the height of the controversy over Megrahi's release, and its full text has now been leaked to the Sunday Times.

In it, the deputy head of the US embassy in London, Frank LeBaron, said the US believed Megrahi should remain in Greenock jail because of the seriousness of his conviction for killing 270 passengers and crew, and 11 Lockerbie townspeople, by bombing Pan Am flight 103 in 1988.

But he added: "Nevertheless, if Scottish authorities [conclude] that Megrahi must be released from Scottish custody, the US position is that conditional release on compassionate grounds would be a far preferable alternative to prisoner transfer, which we strongly oppose."

LeBaron said releasing Megrahi but making him live in Scotland "would mitigate a number of strong concerns we have expressed with regards to Megrahi's release."

Scottish officials took that to mean that the US had only "half-hearted" opposition to Megrahi's release: the embassy comes under the direct control of the US secretary of state, Hillary Clinton, who became the most vigorous critic of Megrahi's release

Allowing Megrahi to live outside prison in Scotland was one of the options considered by MacAskill. Megrahi's wife and sons had a family home paid for by the Libyan government in the prosperous Glasgow suburb of Newton Mearns. But that option was rejected after police advice that this would cause immense security and logistical problems, and cost £100,000 a week to protect him. The house would need a 24-hour armed guard, while Megrahi would need heavy security for his regular trips for medical treatment.

To Salmond and MacAskill's embarrassment, Megrahi is still alive after being allowed home to Tripoli. MacAskill had stated the Libyan had less than three months to live when he was released. They now concede that his life has probably been prolonged from being at home with his family and receiving better medical care.