You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Roxie Gibson
75afe20922
Script is finished!
|
3 years ago | |
|---|---|---|
| .gitignore | 3 years ago | |
| .pylint.rc | 3 years ago | |
| .style.yapf | 3 years ago | |
| Pipfile | 3 years ago | |
| Pipfile.lock | 3 years ago | |
| README.md | 3 years ago | |
| main.py | 3 years ago | |
| requirements.txt | 3 years ago | |
| urls.json | 3 years ago | |
README.md
Landbot.io scraper
Script to scrape a number of landbot.io chat bot messages. This keeps intact choices, dialogue blocks, and image and youtube links.
Installation
Installation requires Python 3.6+, pip, and venv
Also requires Firefox and the Gecko drivers for selenium to interact with Firefox.
git clone REPO_URL
cd landbot_scrape
python3 -m venv .venv
source .venv/bin/activate
python3 -m pip install -r requirements.txt
Running
Add all required urls to scrape to the json file urls.json
To run the script, make sure you are source’d into the virtual env, then run
python3 main.py