"Do you agree that there's a hopeless deadlock that can not be resolved by further deliberations?" he asked after calling the jurors in a little after 10 a.m.
Lawyers for Cosby have said throughout that their client is innocent of all accusations.
Cosby will remain free on $1 million bail. Attorney Gloria Allred, who represents Johnson and many other accusers, said she hoped more can testify at the next trial.
Saturday's result was a victory for the comedian, who avoided up to a decade in prison, but the district attorney said his office would seek a verdict in a second trial.
Cosby's wife, Camille Cosby, slammed prosecutors for bringing the case to court.
She condemned the district attorney as "heinously and exploitatively ambitious", the judge as "overtly arrogant", lawyers for the accusers as "totally unethical" and "many" in the media as "blatantly vicious".
Cosby, who did not take the stand during his trial, did not comment outside court. Constand said after she took the pills she became foggy and unable to fend off Cosby's alleged sex assault after he led her to a sofa. Cosby is charged with sexually assaulting Andrea Constand at his home in suburban Philadelphia in 2004.
As the mistrial was declared, Cosby had a flat, blank look on his face. Over two visits, she said, he touched her "without warning" and tried to unbutton her trousers before she stopped him.
Much has been said and written about the sex scandal that has torpedoed Bill Cosby's legacy - so much so that it can be hard to recall how he reached the point of facing trial on three counts of felony aggravated indecent assault.
The jury could not reach a verdict in the trial of Bill Cosby.
More than 50 women claim Cosby sexually assaulted them in incidents dating back to the 1960s, when he emerged as a major comedy star. The jurors have been trying to break a deadlock.
Prosecutors made the decision to try Cosby again - even before finding out how the jury split on the 3 counts of sexual assault.
During their deliberations, jurors sent a string of questions to the judge, asking to rehear the testimonies of five trial witnesses and Cosby's previous deposition testimony.
And America's Dad, as he was once known, allegedly had very specific tastes. They repeatedly asked a judge for a mistrial on Friday.
Judge Steven O'Neill refused, saying there's no precedent to halt the jury's work.
Jurors began deliberating late on Monday (June 12), as Vulture reports.