Background: Physical activity has known benefits during pregnancy; however, the optimum volume of physical activity through the different stages of pregnancy is not well known. Objectives: The aims of this study were to investigate the associations of physical activity volume in pregnant women in each trimester of pregnancy with maternal and infant outcomes. Design: The study involved 1657 pregnant women from the Australian Longitudinal Study on Women’s Health, who completed surveys from 2006 to 2012 (aged 28–39 years). Methods: Women reported being in either the first, second or third trimester of pregnancy. Women were grouped into four groups according to their self-reported physical activity during pregnancy: (1) Nil (0–<33.3 MET.min/week), (2) Low (33.3–<500 MET.min/week), (3) Moderate (500–<1000 MET.min/week) and (4) High (⩾1000 MET.min/week). Women who reported their physical activity during pregnancy completed a survey within three years after the birth, relating to outcomes associated with pregnancy and childbirth (gestational diabetes, hypertension, and antenatal depression and anxiety) and infant outcomes (birthweight and prematurity). Results: There was no association of physical activity in any trimester with infant birthweight, prematurity, gestational diabetes, hypertension or antenatal depression. Antenatal anxiety was less prevalent in women who reported low (1.7%) or moderate (1.1%) physical activity than in those who reported no activity (4.7%; p = 0.01). Conclusion: Different amounts of physical activity during pregnancy were not associated with the measured adverse health outcomes. However, low and moderate amounts of physical activity were associated with reduced incidence of antenatal anxiety.