The agreement should protect the government from being brought down by a vote of no confidence, but they will have to agree other issues on a vote-by-vote basis.
Foster and her colleagues were expected to ask for concessions on several Tory policies, including the scrapping of the triple-lock pension scheme, but May has said the NI party will have no veto on major policies.
But the source confirmed there was no need for a deal on a so-called "confidence and supply" arrangement to be sealed in order to press ahead with the Speech, and said Mrs May wanted the Government to "get on with its business".
But within the Conservative Party, the election recriminations are continuing.
British Prime Minister Theresa May neared a deal with a Northern Irish Protestant party to save her premiership on Tuesday but faced a tug of war over her Brexit strategy just days before embarking on formal divorce talks with the European Union.
The prime minister may not be present as the talks continue because she is heading to Paris for a meeting with newly-elected President Emmanuel Macron.
She said: "As we welcome new members on all sides, we should celebrate the fact that we have a record number of MPs from Black and Minority Ethnic backgrounds, including a female Sikh MP, we also have more disabled MPs and more LGBT MPs".
There is also concern that Mrs May's planned deal with the DUP could threaten peace in Northern Ireland, as the Government could no longer claim to be impartial between the unionist community which the DUP represents and the the republican and nationalist community.
"Any deal which undercuts in any way the process here or the Good Friday Agreement is one which has to be opposed", he said.
There's been a lot of hyperbole talked about our position to the gay community.
While the DUP are deeply eurosceptic, they have baulked at some of the practical implications of a so-call hard Brexit - including a potential loss of a "frictionless border" with the Republic of Ireland - and talks will touch on efforts to minimise the potential damage to Northern Ireland.
His comments came after Northern Ireland's main parties held talks with Prime Minister Theresa May in Downing Street.
Divisions over Europe helped sink the premierships of Margaret Thatcher, Major and Cameron, and many of her lawmakers and party membership support a sharp break with the EU.
- Monaco's Kylian Mbappe to 'sort through' transfer options over summer
- Feliciano Lopez to face Lucas Pouille in Stuttgart final
- Schiff says firing Mueller would echo Watergate
- Afghan soldier attacks foreign soldiers, wounding 4
- Mining company BHP chooses former Amcor chief executive to be chairman
- Bill Cosby's Wife Camille Releases Statement Thanking Jury
- Anniversary of the death of Jo Cox MP
- Junket tourism
- Odd Market Reaction to Hawkish Fed Has Two Explanations
- Officials Urge to Stay Cool as Heat Wave Envelops SoCal