Since Rocksmith+ is a game that will get content added continuously, and that I do scrape it, I figured I could also do with some analytics. The past weeks I have been writing a somewhat crude analyser, first to see if I could make some nice charts, and then to find some interesting data to display.
Now let us get to the basic first. WordPress have a REST API that allows me to create posts by generating text in scripts or apps, and inject them to this blogg. Since Python3 was the best place to start setting up the analytics I went there.
Second I had to connect to the database where I have the scrape stored, luckilly I already had that covered from the scraper, as I do use python in the process of getting large amounts of data into the database.
Third was to make charts. Python matplotlib is the place to go, and with some assistance from numpy I could get various things done. I made a few functions for the different chart presentations that interests me to begin with, and got them to store the output images as
Creating database queries for the different datasets of interest is not as easy as it sounds. The SQL calls themselves are not very complicated, but to compile the data composition isn’t easy. What data is of any significance? How do I present this data? Should I include all the data or just a subset? For some of these it became quite clear imediately that a subset was required, while others didn’t make sense if I removed any parts of the data.
Visual presentation of the charts are just as important, if not more important, than the data itself. I wanted a certain amount of dynamics to it, without it becoming chaotic. I think I got it down just right. This part will keep evolve as I get feedback on how the data is found useful, what data is missing, and how I represent the data.
The data should be scheduled to be posted quite soon, I hope you get some useful information out of it. If there are anything from the scrape you are curious to see visualized, let me know, and I will try to update the analytics with more information as time goes.