-
Error handling added · 662b9e66serpucga authored
The system should now be capable of overcoming a failure during the process of conversion, either by ignoring the error or by dumping the state at the moment of failure and allowing to resume the process later from the point where it was stopped. The policies followed at this stage for avoiding corrupt data or other errors are the following: 1. If an specific tweet raises an error in the process of being converted to CSV, the tweet is skipped and the whole execution continues. 2. If there is any other error when processing a page of tweets, the number of that page is recorded in the recovery file, and that page will be skipped when the user tries to resume the execution from the recovery file. 3. If any other unexpected error, keyboard interruption or anything happened, a standard recovery file will be dumped, with the list of already converted pages but without "error_page", so when executing the script with the flag "-r", the program will try to resume the execution from the point where it was left without discarding any info.
662b9e66
Name |
Last commit
|
Last update |
---|---|---|
config | Loading commit data... | |
lib | Loading commit data... | |
.gitignore | Loading commit data... | |
pymongoexport_csv.py | Loading commit data... | |
pymongoexport_json.py | Loading commit data... | |
requirements.txt | Loading commit data... |