Presidents’ Day honors the two greatest presidents in U.S. history: George Washington and Abraham Lincoln. Sometimes Americans use the occasion to extend attention to other important presidents, too, like Franklin Roosevelt or Dwight Eisenhower.

But what about the other other guys: the presidents who showed potential, but never got their chance? This Presidents’ Day, let’s bring out of the shadows the might-have-beens of presidential history, and three of them in particular: Zachary Taylor, James Garfield, and—yes—Gerald Ford.

Taylor last made news in 1991, when his body was exhumed to test the theory that his death after 16 months in office could be attributed to arsenic poisoning. (Answer: no.) Otherwise, Taylor seems nowadays to exist mainly to test the memory of AP history students, who must recall whether he came before or after Benjamin Harrison.

But what if he’d lived longer?

Taylor, a Louisiana plantation owner, was the last American president to own slaves. In fact, his slaveholding was one of his major qualifications for office: it assured Southerners that this Whig could be trusted not to go abolitionist on them.

His leading qualification, of course, was his generalship in the war against Mexico, culminating in the victory at Buena Vista, where he defeated a Mexican Army that outnumbered the U.S. forces more than 3 to 1. And once elected, Taylor revealed himself as willing to subordinate his views on slavery to his nationalism. To allay Northern fears that the victory over Mexico would shift the national political balance in favor of slaveholding, he accepted the Wilmot Proviso, which would have barred slavery from all the territories taken in the Mexican war—and which sadly failed to emerge from Congress. He also endorsed admitting California and New Mexico as free states.

Taylor’s premature death removed from national politics the commander who would most have frightened Southerners contemplating rebellion. (He had threatened to hang rebels against the Union less reluctantly than he’d hanged Mexican spies.)

Well, maybe the Civil War could not have been prevented, not even by Old Rough and Ready, as Taylor was nicknamed. But once the war had ended, the country faced another question: what (if anything) should the federal government do on behalf of the freed slaves? The first three postwar presidents—Andrew Johnson, Ulysses Grant, and Rutherford Hayes—offered little or nothing. The fourth, Garfield, was different.

Garfield’s inaugural address in 1881 outlined the most ambitious program on behalf of Southern blacks that would be heard from any president until Lyndon Johnson. He endorsed fully equal political rights for Southern blacks and whites, a radical idea in the postwar era. He even proposed federal support for primary education for Southerners, white and black—a proposal of which little would be heard again for the next 85 years.

We’ll never know whether Garfield could have succeeded where his predecessors and successors opted not to go. He was shot by a deranged assassin four months into office. What we can say about him is only this: he had courage, which he demonstrated during the Civil War at the bloody Battle of Chickamauga, and he had considerable political skill, having served for nearly two decades in Congress.

The third might-have-been president, Ford, served longer than either Taylor or Garfield: 29 months. That may seem long enough to give an idea of what the full presidency would have been like. After all, John F. Kennedy left a strong memory behind after 30 months. But the Ford presidency never had the chance to set a course for itself. Ford took office as the United States plunged into what was then the worst recession since the Great Depression. Defying then-prevailing economic theory, this recession was accompanied by high inflation. Drugs, crime, divorce, defeat in Vietnam: the country seemed to be falling apart. The presidency could only react. Ford reacted well, with an agenda of deregulation and tax reduction that anticipated the Thatcher and Reagan revolutions of the next decade. But facing one of the most lopsided, hostile congressional majorities ever confronted by a president, Ford’s agenda mostly stalled.

Economic recovery would begin in 1977, and with it a chance to govern. That chance would accrue to Jimmy Carter, who notoriously bungled it. Had a few thousand votes in Ohio or Hawaii gone the other way in 1976, the chance would have been Ford’s—and this president praised for his decency by friend and opponent alike might also have been praised for his accomplishments.

As is—we’ll never know. But for Ford and the other commanders in chief whose time in office was cut too short, we can at least spare—if nothing else—a Presidents’ Day moment of speculative remembrance.