Rocksmith+ Scraper

If you are actively using my Rocksmith+ Songs database in any way, but feel your region is unreliable, missing, or not updated often enough, you can help out.

The more people assisting in the scraping, from more places, the better the data gets. I have prepared this by preparing a docker container that will execute the scrape. The container is actively maintained as features are added, or changes in the API done, so that we can harvest data in the most efficient way.

I assume you have installed docker on your system, this will not be a tutorial in how to install docker or what you can do with docker. If you are familiar with how docker works and how to set it up on your system, just pull the latest image from ghcr.io/skippern/rsplus-scraper:latest . I would recommend scheduling the job, preferably daily, but any schedule that fits you better is also apriciated.

I myself have added the following to /etc/crontab on my Ubuntu linux:

0 0 * * * skippern docker pull ghcr.io/skippern/rsplus-scraper:latest && docker run -d ghcr.io/skippern/rsplus-scraper:latest

You can also have an instance run continuous with docker compose using the following docker-compose.yaml-file. Save it anywhere and run docker compose up -d from the location where you saved it. I also do run an instance in this way.

version: '3.7'


services:
  rsplus-scraper:
    deploy:
      restart_policy:
        condition: any
        delay: 60s
    image: ghcr.io/skippern/rsplus-scraper:latest
    cap_add:
      - NET_ADMIN
      - SYS_ADMIN
    networks:
      - netvpn
    

networks:
  netvpn:
    name: netvpn

This page is work in progress, and will be updated as the scraper develops.

Running Docker on Windows11

My best suggestion is to run docker through Windows Subsystem for Linux (WSL). There might be other solutions, maybe even better. I used the following commands for a test:

wsl --set-default-version 2
wsl --install -d ubuntu

You will be prompted to create a UNIX user and a password, remember the username and password, they will come handy. After reboot, start wsl and run the following commands:

sudo snap install docker
sudo docker pull ghcr.io/skippern/rsplus-scraper:latest
sudo docker run -d ghcr.io/skippern/rsplus-scraper:latest

Running Docker Desktop on Windows11

Thanks to user NotMyself I got this instruction for using Docker Desktop on Windows 11

Ensure you meet the system requirements for running the Windows Subsystem for Linux.
Open PowerShell
Install WSL, detailed directions
1. Run the command wsl --install
2. Ensure you are running WSL 2 by running the command wsl --set-default-version 2
Close PowserShell
Install Docker Desktop, detailed directions
1. Ensure you check the box to use wsl in the installer
Open PowerShell
Ensure docker cli is available, docker --version
Pull the image, docker pull ghcr.io/skippern/rsplus-scraper:latest
Run the image in a container, docker run -d ghcr.io/skippern/rsplus-scraper:latest

From here, you can run docker ps and see the container is up and runnning.

Running Docker on Windows10

I have had feedback that the Windows 11 instruction might work on Windows 10, I have not been able to replecate that in Windows 10. At current time I havn’t been able to get docker working in Windows 10.

Running Docker on Mac

Before starting you need to download two things, Docker Desktop for Mac, and MacPorts (you could probably do a similar approach with HomeBrew, but I am familiar with MacPorts). After installing Docker Dektop and MacPorts, run the following commands in a terminal:

sudo port selfupdate
sudo port install docker

Now you need to open Docker Desktop, you only need it for the docker demon. You will be asked several questions during the first start. You DO NOT need to register an account with Docker Hub, the following commands will work without login:

sudo docker pull ghcr.io/skippern/rsplus-scraper:latest
sudo docker run -d ghcr.io/skippern/rsplus-scraper:latest

Updating the Docker image

This description should be the same cross platforms. If the docker image does not renew by itself, run the following commands. This will remove the current image and pull a new docker image from the hub before running it:

sudo docker rmi -f ghcr.io/skippern/rsplus-scraper:latest
sudo docker pull ghcr.io/skippern/rsplus-scraper:latest
sudo docker run -d ghcr.io/skippern/rsplus-scraper:latest

Gimnechiske.org