Antimicrobial resistance is a growing public health threat predicted to cause up to 10 million deaths a year by 2050. To circumvent existing bacterial resistance mechanisms, discovering antibiotics with novel modes of action (MoAs) is crucial. While growth inhibition assays can robustly identify antibiotic molecules, they miss promising compounds with subinhibitory phenotypes and do not inform on drug MoA. Microscopy-based cytological profiling of drug-treated bacteria with hand-crafted image descriptors or more recently deep learning (DL) provides complementary information on the MoA. However, current approaches are limited by the need for fluorescent labelling and drug exposure at inhibitory concentrations. It also remains unclear if cytological profiling enables the detection of drugs with novel MoAs. Here, we demonstrate an approach based on supervised DL to identify antibiotic drug MoA from microscopy images without fluorescent labelling. We train a convolutional neural network to predict treatment conditions from brightfield images of Escherichia coli exposed to reference drugs covering multiple MoAs. Our method can detect drug exposure at subinhibitory concentrations and distinguishes individual drug treatments with high accuracy (86.2%). The learned representations implicitly capture MoA-specific phenotypes, enabling perfect MoA recognition (100%) without retraining using a model trained on only 644 images. Our approach can identify the MoA of previously unseen drugs with good accuracy (77.8{+/-}3.3%), as long as the MoA is represented by at least one of the training compounds. Finally, we show that our approach can detect if a drug has a novel MoA with an area under the curve above 0.75 for five out of six MoAs, facilitating microscopy-based identification of novel classes of antibiotics. Our methods and results pave the way towards an automated pipeline for antibiotic drug discovery based on imaging and DL.