Hello,
As I understood you need a bot which will render the pages of a website from sitemap and record the metrics to a file.
I'm a software developer working on PHP and NodeJS environments for over 5 years, I expertise in web scraping and automation.
There are 2 ways to go around this project:
1. Handle everything with HTTP calls.
2. Use headless browser like PhantomJS or Headless Chrome to actually render the page and process javascript.
The 2nd option would involve more heavy processing and bandwith usage, but we could use service like [login to view URL]
If the first option is acceptable for your use-case, then the project is fairly simple. I'd use Async and Request modules in NodeJS to build a CLI tool which will take standard sitemap XML and visit the pages to figure out what are all the metrics you are looking for.
I also have a few optimizations ideas, mostly based on statistical analysis of the already gathered data.
Let's have a quick chat and go over all the details.
Best,
Nick.