After reading the linked paper, I find it only mildly worrying:
Normal limitations pertaining to inferences about cause and effect from cross-sectional studies apply3,20. One might posit that people with lower cognitive ability have higher risk of catching the virus. We consider such a relationship plausible; however, it would not explain why the observed deficits varied in scale with respiratory symptom severity. We also note that the large and socioeconomically diverse nature of the cohort enabled us to include many potentially confounding variables in our analysis. Nonetheless, we emphasise that longitudinal research, including follow-up of this cohort, is required to further confirm the cognitive impact of COVID-19 infection and determine deficit longevity as a function of respiratory symptom severity, and other symptoms. It also is plausible that cognitive deficits associated with COVID-19 are no different to other respiratory illnesses. The observation of significant cognitive deficit associated with positive biological verification of having had COVID-19, i.e., relative to suspected COVID-19, goes some way to mitigate this possibility.
After reading the linked paper, I find it only mildly worrying: