It is commonly thought that drug addiction involves a transition to habitual control of action, where the choice to consume drugs becomes automatized and reflects a failure to deliberate over possible negative outcomes. Determining whether the pursuit of addictive drugs is habitual is hampered by a lack of behavior assessments suitable for use during a bout of drug seeking. Therefore, to understand how variable histories of drug reinforcement might affect goal-directed and habitual pursuit of drug rewards, we trained rats to perform a multi-step decision-making task to earn oral fentanyl or sucrose rewards following extensive training with either fentanyl or sucrose. Importantly, this task allowed for independent measurements of goal-directed and habitual choice characteristics during online pursuit of rewards, and habitual choice could further be categorized into perseverative and reward-guided components. Extensive fentanyl training led to a bias for reward-guided habitual choice specifically in females, and a relatively high degree of perseveration in both sexes. These behavioral changes after chronic fentanyl generalized across fentanyl and sucrose seeking. In contrast, during fentanyl sessions, females showed increased perseveration, and both sexes showed blunting of the gradual improvement in goal-directed choice. These results show that chronic drug reinforcement induced habits that generalized across seeking drug and non-drug rewards, and female rats were especially susceptible to habitual control induced by both chronic and acute drug reinforcement.