Generative artificial intelligence use in automated urban ecological assessments requires substantial human oversight

• Automated data processing and generative AI could scale ecological assessment across urban areas. • We tested the quality of automated ecological reports for 39 urban areas. • The reports were assessed to require substantial human input and editing. • Automation could reduce the labour cost of reporting by 10%. • The costs of writing urban ecological reports for planning are likely to remain prohibitively high. Automated data processing pipelines and generative artificial intelligence (AI) present new opportunities for scaling ecological assessments across urban areas, yet the practical utility and limitations remain untested. This study provides a workflow for automated urban ecological reporting, which integrates 25 public datasets and performs statistical and spatial data analyses to quantify indicators of biodiversity and ecosystem services. The workflow incorporates large language models to aid synthesis and writing. Reports were generated for diverse cities worldwide and reviewed by domain experts to assess quality, trust, and potential to inform urban planning. Respondents found that while the structure and data integration had potential to be helpful, the draft reports required substantial human revision. Factual sections relying on high-quality datasets needed the fewest changes, whereas content based heavily on AI inference, such as descriptions of climate change adaptation options, were inaccurate, generic, or culturally inappropriate. Despite these limitations, participants generally viewed the reports as potentially helpful. Of the total labour required to create reports, respondents estimated that around 10% could be substituted by automation. Our findings suggest that AI-assisted automated report generation may be scaled up to support urban sustainability efforts, but only with strong human oversight and transparent disclosure of AI use. Trust in automated assessments depends on transparency, and the inclusion of local voices in legitimising final outputs. Even with automation, substantial investment in human labour will be required to make ecological assessments available for towns and cities around the world.