That information included details on Americans' phone calls such as the numbers they were calling and the duration of calls but not the content.
The decision also recognizes that when the government seeks to prosecute a person, it must give notice of the secret surveillance it used to gather its evidence.
The metadata collection program was created and approved by the secretive Foreign Intelligence Surveillance Court after the passage of the Patriot Act in the wake of the 9/11 terrorist attacks.
The court's decision Wednesday, which was written by Judge Marsha Berzon, a Clinton appointee, ruled that the bulk collection of phone records violated the requirement that agencies seek a court order when obtaining information relevant to an investigation from a private business.
The court stopped short of ruling the metadata program unconstitutional, saying it was irrelevant to the defendants' convictions.