Guilt's impact and overall relevance went largely unrecognized during their six-year tenure as an intense and emotive über-indie-noise-metal-hardcore band.

Their songs combined the best elements of several genres without sounding contrived and managed to evoke soulful emotion and raise thought-provoking questions within the confines of a musical style typically confined to mundane "evil" topics and horror imagery.

Guilt evolved from the band Stepdown, which formed in 1991 with former Endpoint drummer Lee Fetzer and Endpoint bassist Kyle Noltemeyer on guitar, Christian McCoy on bass, powerhouse drummer Jon Smith, and Endpoint guitarist Duncan Barlow on vocals. Stepdown were fiercely political and rabidly confrontational, going head to head with the Nazi faction of the Louisville scene on more than one occasion. As Barlow's own personal battle with depression set in, however, the group's focus shifted and soon the name was changed. Fetzer left the group and Barlow picked up second guitar in addition to vocal duties. The Empty 7" was recorded at Mom's Studio and released by Initial, a small record label then based in Michigan. After a handful of shows, the band dissolved for almost a year, eventually re-forming and recording several new songs at DSL Studios that would later become the Synethesia 10"/CD EP on Initial. The record was heavy and intense, with darkly poetic lyrics and song titles named after colors picked to represent various moods. McCoy was replaced by Telephone Man's Ashli State before the band signed a deal with Chicago's Victory Records label.

Guilt entered the studio with producer/Shellac member Bob Weston, who helped the band hone their sound by enhancing the drums, bringing down the metal element in the guitars, and eliciting top-notch, emotional performances. The end result was Bardstown Ugly Box, the band's masterwork, so named for a street in their native Louisville, KY. It was years ahead of its time in the hardcore/punk scene, fusing elements of noise, melody, heavy metal, punk rock, indie rock, and poetic storytelling seamlessly, with intensity yet class

For the 2023 reissue of this classic album, Mind Over Matter Records worked closely with the band as they revisited the audio and preserved that energy & effort into the physical release. The digital remix was completed by Steve Evetts (Dillinger Escape Plan, Every Time I Die), it was mastered by Brian Luken and then carefully cut into lacquer form by Dave Eck at Lucky Lacquers. With two newly unearth bonus tracks, MOMR also added in a bonus 7" to ensure that purists can continue to enjoy Bardstown Ugly Box as they've always known it and everyone gets the opportunity to hear previously unavailable tracks separate from the original track listing.