ASHINGTON, June 16 — The Supreme Court placed strict limits today on the government's ability to medicate mentally ill defendants forcibly to make them competent to stand trial.

The use of antipsychotic drugs must be in the defendant's best medical interest and "substantially unlikely" to cause side effects that would compromise the fairness of the trial, the court said in a 6-to-3 decision by Justice Stephen G. Breyer. Additionally, the court held, the government's "important" interest in bringing the defendant to trial must be unattainable by alternative, less intrusive means.

Whether the defendant has been accused of a violent or, as in this case, a nonviolent crime, the circumstances permitting the involuntary administration of antipsychotic drugs "may be rare," Justice Breyer said.

The decision overturned a ruling by the federal appeals court in St. Louis authorizing federal prison officials to medicate forcibly a dentist charged with Medicaid fraud and mail fraud. The dentist, Dr. Charles T. Sell, has been found incompetent and has refused medication, resulting in a legal standoff that has kept him confined without trial for more than four years in the federal prison system's medical center in Springfield, Mo.

This long incarceration and the nonviolent nature of the charges have combined to make Dr. Sell in some quarters a symbol of clinging to personal autonomy in the face of overweening government power. The case also divided the mental health profession, with the American Psychiatric Association supporting the government and the American Psychological Association filing a brief on behalf of Dr. Sell.

He has been found to have "delusional disorder, persecutory type" and believes that federal agents, particularly those of the Federal Bureau of Investigation, are conspiring against him. In a separate case that was not before the court, he was indicted in 1998 on charges of attempting to murder the F.B.I. agent who arrested him on the fraud charges. Nonetheless, the lower courts in the fraud case determined that Dr. Sell, while confined, was not a danger to himself or others, and Justice Breyer said today that the Supreme Court was obliged to accept that conclusion for purposes of deciding the case.

The court's starting point was a premise established by two related decisions during the 1990's: that a criminal defendant has a basic, constitutionally protected "liberty interest" in avoiding unwanted antipsychotic medication that can, however, be trumped by the government's interest in bringing a defendant to trial on serious charges. But neither of those cases directly addressed the issues posed by Dr. Sell's case: how to strike the balance when the defendant has been charged with a nonviolent offense and has been found not to present a danger while confined in an institution.

While a victory for Dr. Sell, the decision, Sell v. United States, No. 02-5664, gave the government another chance to try to persuade the lower federal courts that it can satisfy the multipart test the court established today. But Justice Breyer suggested that one factor in the equation, the government's interest in bringing Dr. Sell to trial, may have diminished, given that "a defendant ordinarily receives credit toward a sentence for time served." Dr. Sell's lawyer, Barry A. Short, told the justices when the case was argued in March that the pretrial confinement had exceeded the sentence his client would receive under the federal guidelines if he were convicted.

The dispute among the justices concerned the pretrial nature of the appeal. The lower courts permitted Dr. Sell to challenge the medication order in advance of his trial, under an exception to the usual rule that requires issues to be reserved for appeal after a conviction. The Supreme Court accepted the case and the majority proceeded to decide it on that basis. "By the time of the trial Sell will have undergone forced medication — the very harm that he seeks to avoid" and that he would suffer even if eventually acquitted, Justice Breyer said.

In a dissenting opinion today, Justice Antonin Scalia objected that the trial should have gone forward, with Dr. Sell's challenge to his involuntary medication put off until an appeal after a conviction. To permit pretrial appeals of this sort, Justice Scalia said, provides an "obvious opportunity for gamesmanship" and invites "the disruption of criminal proceedings" by defendants challenging a variety of pretrial orders. For example, he said, "an order refusing to allow the defendant to wear a T-shirt that says `Black Power' in front of the jury could be attacked as an immediate violation of First Amendment rights."

Justices Sandra Day O'Connor and Clarence Thomas joined Justice Scalia's opinion, which did not express a view on the medication issue itself. The part of the opinion permitting defendants to raise the medication issue before trial could prove to be of significant benefit to defendants in such cases.

In his opinion, Justice Breyer summarized by this question the steps that the court was requiring for the protection of defendants' rights to due process: "Has the government, in light of the efficacy, the side effects, the possible alternatives, and the medical appropriateness of a particular course of antipsychotic drug treatment, shown a need for that treatment sufficiently important to overcome the individual's protected interest in refusing it?"

Joshua Dratel, a defense lawyer in New York who worked on a brief on Dr. Sell's behalf for the National Association of Criminal Defense Lawyers, said in an interview today that the court's concern for the potential side effects of antipsychotic medication should prove beneficial to defendants.

"It is widely underappreciated that the defendant has a right to be functional at his trial," Mr. Dratel said. He explained that medications that interfere with alertness or that cause headaches, rashes, or other distracting problems can compromise the ability of defendants, even those who are technically competent to stand trial, to help their lawyers and to make a favorable impression before the jury.

Another lawyer who filed a brief for Dr. Sell, Judy Appel, deputy director of legal affairs for the Drug Policy Alliance, found the outcome of the case less reassuring. Ms. Appel said the decision was a mixed bag that "gives prosecutors the tools to ask for, and trial courts to impose, a major violation of individual liberty." Noting that Justice Breyer predicted that involuntary medication would be rare under the standards the court set today, she said, "What the court predicts is not necessarily how things play out at the trial court level."

Another forcible medication issue will soon come before the court. Charles L. Singleton, a death row inmate in Arkansas, has appealed a ruling this year by the same appeals court, the United States Court of Appeals for the Eighth Circuit, permitting his forcible medication to make him sane enough for execution. The justices have not decided whether to accept his appeal.