I wanted to listen to some older CarTalk episodes but they did not have a readily available archive. This sent me down a rabbit hole on trying to find these old episodes. I was going through some Reddit posts and found a URL that showed about a month's worth of episodes from 2018, and I started messing with the URL and the packets/requests sent to the webserver. Eventually, I figured out how to modify the URL to scroll through the different months, but it was still tedious messing with the URL. So, I built a Python script to download all the possible combinations of that URL and then combine them, then pull out the link that actually links to the episode. Using that, I was able to build this site!
1. Download my downloader script
2. Install dependencies
pip install beautifulsoup4 requests tqdm lxml3. Run it
python3 fetch.py4. Download this script and paste it in the HTML folder
5. open terminal in the HTML directory. Then run it.
python3 combine.py6. Download this script and put it in the html directory.
7. in the same terminal run the 2nd script
python3 list.py8.You can finally run the download, Download this script and copy it to the html folder
9. Run the last script!
python3 download.pyNote: It is normal for it to downlaod a few eppesodes then throw a error! if it does this just rerun the download.py agin!
Note: It is normal to be stuck on a blinking cursur for a while! The script is not stuck!
Now all the epesodes should be in the audio folder!
To add on new epesodes do not delete anything and just run fetch.py again in the same folder you previously ran it in (probobally downloads) then go through all the steps again but using the scripts you already have. (you dont need to redownload the scripts!)