Anxiety spectrum disorders are characterized by excessive and uncontrollable worrying about potential negative events in the short- and long-term future. Various reports linked anxiety spectrum disorders with working memory (WM) deficits despite conflicting results stemming from different study approaches. It remains unclear, however, how different anxiety spectrum disorders such as generalized anxiety disorder (GAD), social anxiety disorder (SAD), and panic disorder (PD), differ in WM function. In this study, we utilized verbal, numerical, and sequential evaluations of WM to cover most possible facets of the WM data space. We used principal component analysis to extract the uncorrelated/whitened components of WM based on these measures. We evaluated medication-free patients with GAD, SAD, and PD patients as well as matched healthy individuals using a battery that measures WM duration and load. We found that patients with GAD and SAD, but not PD, exhibited poor performance only in the WM principal component that represents maintenance. There were no other significant differences between the four groups. Further, different WM components significantly predicted the severity of anxiety symptoms in the groups. We explored the clinical utility of WM components for differentiating patients with anxiety spectrum disorders from healthy individuals. By only using the WM components that represent maintenance and encoding, we managed to differentiate patients from controls in 84% of cases. For the first time, we present multiple novel approaches to examine cognitive function and design cognitive screening, and potentially diagnostics, for psychiatric disorders.