Physical activity levels decrease during pregnancy, and the time course of return to prepregnancy levels is unclear. This study aimed to describe changes in leisure-time physical activity (LTPA) and sitting time from 16 to 24 weeks of pregnancy to 12, 24, and 48 months postpartum in women with different education levels in Brazil. Methods: Data from 4000 mothers of children enrolled in the 2015 Pelotas (Brazil) Birth Cohort were analyzed. The women were interviewed between 16 and 24 weeks of pregnancy and when their children were aged 12, 24, and 48 months. The LTPA and sitting time were self-reported. Results: Only 15.7% of the women reported any LTPA during pregnancy; this declined to 7.9% at 12 months postpartum; it was 16.8% at 24 months and 23.2% at 48 months. On average, participants spent a mean (SD) of 6.4 (3.9), 4.2 (3.2), 4.3 (3.3), and 4.4 (3.3) hours per day sitting during pregnancy, and at 12, 24, and 48 months after the birth, respectively. Both any LTPA and high sitting (8+ h/d) were consistently higher among women with higher education. Conclusion: After 24 months postpartum, LTPA levels had returned to or exceeded pregnancy levels, but sitting time remained lower than during pregnancy.