The police said they received a report about the shooting in the Todt Hill section of Staten Island around 9:20 p.m.
The assailant sped off in a pickup truck, police said.
This is the first hit on a Gambino boss since Paul Castellano was gunned down in 1985 in front of Sparks Steak House.
Following his death, notorious mobster John Gotti was named the new head of the crime family until 1992 when he was found guilty on multiple charges including murder, conspiracy to commit murder, bribery and tax evasion.
Police work near the scene where an alleged leader of the Gambino crime family was shot and killed in the Staten Island borough of New York, Thursday, March 14, 2019.
Cali served a 16-month prison sentence for the crime.
The Gambino family is one of New York's five historic Italian Mafia families - along with the Genoveses, Luccheses, Colombos and Bonannos.
NY media say it is the first targeted killing of a mob boss in the city since 1985.
The Gambino family was once among the most powerful criminal organizations in the US, but federal prosecutions in the 1980s and 1990s sent Gotti and other top leaders to prison, diminishing its reach.
"We thought those days were over", Mayor Bill de Blasio said of Cali's slaying.
He began ascending within the ranks under capo John D'Amico, who went by Jackie and became the family's acting boss in 2005 - after which Cali became an acting captain, according to court documents. "But I guess old habits die hard". One neighbor told the New York Times he heard a burst of about seven gunshots.
Last October, 71-year-old Sylvester Zottola, a reputed associate of the Bonanno organisation, was shot dead at a takeaway restaurant in the Bronx, New York.
A week ago, Persico died after serving 33 years in prison. At a 2008 bail, a prosecutor said Cali was seen "as a man of influence and power by organised crime members in Italy".
- Christchurch shooting: Terrorist Brenton Tarrant produced before court, charged with murder
- Ivanka Trump insisted her dad was 'not a racist,' new book says
- Weapons used in Christchurch, NZ attack appear to have been modified: Ardern
- Patriots reportedly showing interest in free-agent WR Golden Tate
- Facebook says Chief Product Officer Chris Cox is leaving | Tech
- Manchester United boss Solskjaer rejects Neville's remarks on priorities
- Can Manchester United win the Premier League this season?
- Amazon Fire TV finally gets Apple Music streaming
- Tragic Love Island star Mike Thalassitis recently lost his beloved nan
- May counts on European Union for Brexit delay, World News & Top Stories