Clarence Aaron was a 23-year-old college student from Mobile, Alabama, with no criminal record. In 1992, he introduced a classmate whose brother was a drug supplier to a cocaine dealer he knew from high school. He was subsequently present for the sale of nine kilograms of cocaine and was paid $1,500 by the dealer. After police arrested the group, the others testified against Aaron, describing him as a major dealer, which led to his being sentenced to three terms of life imprisonment.

Unfortunately, in the era of harsh mandatory sentencing laws, stories such as Aaron’s are all too familiar. The injustice against Aaron was eventually recognized and, in 2013, after 20 years in prison, he became one of a relative handful of federal prisoners to receive a sentence commutation from President Obama. Cases such as his have fueled momentum for criminal justice reform in recent years, with major presidential candidates in both parties calling for a substantial reduction in our prison population, due to a U.S. rate of incarceration that’s five to ten times that of other industrialized nations. A growing consensus has developed around the idea that the “war on drugs” has relied far too heavily on excessive punishments, and that treatment interventions for substance abusers are both more effective and compassionate than long-term imprisonment.

But if a prison-reduction strategy is focused primarily on drug policy reform, we will be sorely disappointed in the results. Of the 2.2 million people behind bars in America today, nearly half a million are incarcerated for a nonviolent drug offense. So even if we were to release that entire group, we would still have a rate of incarceration far higher than that of any comparable nation.

The heart of the problem, as documented in a major report released by the National Research Council in 2014, is that the tripling of the prison population since 1980 was produced by changes in policy, not crime rates. Half of the prison expansion resulted from sending more people to prison due to the increased adoption of mandatory sentencing policies and prosecutorial charging decisions, while half resulted from longer prison terms. The latter trend is increasingly the major barrier to substantial reductions in incarceration.

Nationally, one of every nine people in prison—160,000 prisoners—is serving a life sentence. About a third are serving life without parole, and of the remainder, political considerations—governors and parole officials believing they need to demonstrate how “tough” they can be on individuals convicted of serious crimes—have made parole release increasingly difficult to secure in many states. In addition, an undetermined number of offenders are serving “virtual life sentences.” For example, a 40-year prison term imposed on a 35-year-old offender essentially equates to life imprisonment.

The excessively lengthy incarceration of offenders—yes, even for violent crimes—is counterproductive, costly, and inhumane. To remedy this problem, Congress and state legislative bodies should establish an upper limit of 20 years in prison as a maximum penalty, except in unusual cases such as a serial rapist who has not been amenable to treatment in prison or a mass murderer. The rationale for such a policy shift is grounded in both humanitarian and public-safety concerns. Life sentences ruin families and tear apart communities; they deprive the person of the chance to turn his or her life around. Moreover, it has long been known that individuals “age out” of crime, and that this occurs at a surprisingly young age. As is true of all adults, offenders mature in prison as they age and develop a longer-term vision for their lives. Research by leading criminologists Alfred Blumstein and Kiminori Nakamura demonstrates that an 18-year-old arrested for robbery is no more likely to be arrested for this crime by the age of 26 than anyone in the general population. Thus, each successive year of incarceration after this decline sets in produces diminishing returns for public safety.

This impact comes at great cost as well. Estimates are that the cost of imprisoning an elderly offender is double that of a young offender, largely due to high health-care costs. Given that public-safety resources are finite, incarcerating aging prisoners inevitably diverts resources from preschool programs, substance abuse treatments, and mental health interventions that all produce demonstrated and substantial crime-reduction benefits.

Lengthy prison terms also exacerbate the dramatic racial and ethnic disparities that have defined the phenomenon of mass incarceration. Nationwide, nearly two-thirds of the people serving life in prison are African-American or Latino. The sight of elderly men of color in prison uniforms and bound in wheelchairs only reinforces the racialized nature of incarceration in the modern era.

Some skeptics would argue that while the public-safety argument may apply to many offenders, there are nonetheless individuals who present such a threat to the community that even 20 years in prison is not sufficient for public protection. That’s certainly correct. But the problem is that on the day of sentencing, no one—including the judge—can predict who those people are, or how individuals may mature over a 20-year period.

For this reason, policymakers could establish a mechanism to evaluate the public-safety risk of select prisoners as they near the end of their 20-year term. A review board comprised of psychologists and other professionals could make recommendations either to a judge or a parole board regarding whether continued confinement is necessary for public safety. And in such cases, they should also propose appropriate treatment interventions designed to produce behavioral change leading to eventual release.

While some might think this is unrealistic, sentences of more than 20 years are quite rare in many democratic nations. Norway, for example, limits prison terms to no more than 21 years, followed by a period of civil confinement when deemed necessary. Even the worst mass killer in the country’s history, Anders Breivik, who killed 77 people in 2011, is serving such a prison term. Contrast this to the current practice in the United States, where countless drug offenders are serving far lengthier terms.

No other industrialized nation incarcerates its citizens at more than a fraction of the rate in the United States. Fewer people are sent to prison in most nations and their terms of imprisonment are considerably less severe. And notably, such policies have not produced spikes in crime. It is long past time to eliminate mass incarceration, and the only way to accomplish that will be to think broadly about how far we have come from any reasonable conception of what a fair and effective justice system should look like.