SAN ANGELO, Texas (CNN) — Texas officials had no right to remove about 460 children from a polygamist sect, the state Supreme Court ruled Thursday.

The Texas Supreme Court agreed with a lower court’s ruling, that Child Protective Services did not present ample evidence that the children were being abused.

The state said it removed the children last month from the Yearning for Zion Ranch near Eldorado, Texas, because in interviews with those living there, officials found what they called a "pervasive pattern" of sexual abuse through forced marriages between underage girls and older men.

The high court ruling could possibly clear the way for the children to be returned to their families. The sect subscribes to the Fundamentalist Church of Jesus Christ of Latter-day Saints, a Mormon offshoot that practices polygamy.

"We are not inclined to disturb the court of appeals’ decision," the ruling said. "On the record before us, removal of the children was not warranted."

The court’s 6-3 ruling came in the case of 38 mothers who had appealed the removal of their children, but attorneys in the case have said the reasoning behind the court rulings can be applied to the removals of all the children from the ranch during the raid, which began April 3.

About 460 children were removed, although 20 were later found in court to be adults.

It’s unlikely the children will be returned to their homes soon because it’s unclear which child belongs to which parent. A DNA testing order by the district court is incomplete.

And even if the children do return to the YFZ Ranch, the case may not be over. The justices noted that Texas law gives the district court "broad authority to protect children short of separating them from their parents and placing them in foster care."

Examples of those conditions might be a court order saying a child must remain in a certain geographical area or an order removing an alleged perpetrator from the child’s home, the district court said.

State officials have said the sect, with its polygamist beliefs, groomed young boys on the ranch to be abusers. FLDS members deny any sexual abuse at the ranch and claim they are being persecuted because of their religion. The district court said in its ruling that the state’s Child Protective Services did not demonstrate that all the children on the ranch faced imminent danger, as necessary for their removal.

The three dissenting justices said in an attached opinion that they agreed the state had no right to remove the young boys from the ranch, but that the district court did not err in electing to remove pubescent girls from the ranch and keep them in state custody.

The pubescent girls are "demonstrably endangered," wrote Justice Harriet O’Neill.

FLDS leader and "prophet" Warren Jeffs, 52, is in a Utah prison, serving two consecutive terms of five years to life after being convicted on two charges of being an accomplice to rape in connection with a marriage he performed in 2001. He also faces trial in Arizona on eight charges, including sexual conduct with a minor, incest and conspiracy.

Jeffs’ trial thrust the normally secretive FLDS into the national spotlight, and the raid on the YFZ Ranch last month intensified scrutiny on the sect. In addition to the YFZ Ranch, the FLDS is prevalent in the twin border communities of Hildale, Utah, and Colorado City, Arizona.

It was not immediately clear whether the Texas Supreme Court ruling would be appealed. In theory, it could be appealed to the U.S. Supreme Court, said CNN legal analyst Jeffrey Toobin, but the court would be unlikely to weigh in because the case relates exclusively to Texas law.

Child Protective Services said it would issue a written statement. Rod Parker, a Salt Lake City, Utah, attorney who has been a spokesman for FLDS families, has scheduled an evening news conference, as has Texas Rio Grande Legal Aid, which represented the 38 mothers.

Texas attorney Barbara Elias-Perciful supported the state in her filing. She has been a guardian ad litem — an attorney representing children’s interests — in child protection cases for 16 years and is also the director of Texas Lawyers for Children.

"This case involves the systematic rape of minor children — conduct that is institutionalized and euphemistically called ’spiritual marriage,’" she wrote. "Typically, there is no media coverage of the horrific acts sexual predators commit against children … if the media showed the actual events of adult males demanding sex with 11-year-old girls, there would be no one questioning the graphic danger of returning these children to their home at this time."