Objective The primary aim of this study was to examine the perceptions of older men with prostate cancer regarding their quality of life and physical activity post-diagnosis, and the potential benefits and risks associated with being physically active. A secondary aim was to gain some preliminary insight into how these perceptions may differ as a function of androgen deprivation therapy (ADT). Methods Two focus groups were conducted, consisting of six ADT and eight non-ADT men, respectively. The probe questions used assessed the link between quality of life and physical activity as well as the benefits and risks associated with physical activity. Data were transcribed verbatim and themes identified using a general inductive thematic approach. Results The primary themes identified were sexual health, 'plumbing' and non-urogenital side-effects, return to and increased levels of physical activity post-diagnosis, physical health/function and psychological benefits of physical activity as well as over-doing it and age-related risks of excessive physical activity. However, not all themes were present in both the ADT and the non-ADT sub-groups. Conclusions These results further highlight the link between physical activity and quality of life in prostate cancer survivors and how they use physical activity as a part of their survivorship process. Of particular interest was how several men on ADT used resistance training to counteract ADT-related side-effects affecting their masculinity. As the evidence for physical activity for prostate cancer survivorship is increasing, cancer clinicians and service providers should consider ways to better assist these men, especially those on ADT become more active.