- 19 Feb, 2019 2 commits
- 14 Jan, 2019 8 commits
-
-
Serbaf authored
-
Serbaf authored
-
Serbaf authored
-
Serbaf authored
-
Serbaf authored
-
Serbaf authored
Fixed the array representation. Before this fix, the arrays of values were represented as "[a, b, c]", which could lead to tremendous bugs due to ambiguity (the example array could have three values, "a", "b" and "c", or only one, "a, b, c", or other combinations). The fixed notation solves that ambiguity, because now the arrays are represented in the form: ["a", "b", "c"]
-
Serbaf authored
-
Serbaf authored
Now the user cannot only specify if he wants to represent the arrays flattened or compressed in the CSV file, but also which depth of compression wih an integer. A level of 0 would equal no compression, a level 1 equals the old compression mode (and the current UPV-compatible mode too), that is, to just unnest 1 level of arrays, for example: house[0].dweller[0]: "Ana" house[0].dweller[1]: "Roberto" house[1].dweller[0]: "Marina" Would become: house[0].dweller: "[Ana, Roberto]" house[1].dweller: "[Marina]" Now, it is also possible to select a compression of depth 2 or more, which would provide the followint output: house.dweller: "[[Ana, Roberto], [Marina]]" Some minor bugs fixed and tests adapted to the new feature. This is the state of the program desired in the beginning of the development, and this version (with possible minor fixes) is the one expected for the 1.0.0 release.
-
- 10 Jan, 2019 4 commits
-
-
Serbaf authored
Since the program has reached a pretty stable stage (to the level provided by automatic and manual testing done by the developer) and with a reasonably good functionality in comparison to its initial goals, the changes are dropped to master. New release version boys!!
-
Serbaf authored
-
Serbaf authored
-
Serbaf authored
Fixed some bugfixes related to the changes introduced at 0.8.0. E.g., in 0.8.0 one could not flatten the split directory of the same collection, because the collection was removed before (the overwriting was performed over the top directory of the collection, and thus the input for the operation was removed before beginning to flatten). Tests have been adapted too
-
- 09 Jan, 2019 2 commits
-
-
Serbaf authored
-
Serbaf authored
This patch includes a big reestructuration of the directory hierarchy produced by the program. The three standard folders for output (split\_tweets, flattened\_tweets and csv\_converted\_tweets) cease to exist. Now the only preexistent folder for output is "output" itself. Everytime a collection is split, flattened or converted, a new output folder with the name of that collection is created inside "output". These collections can have up to three subdirectories called "Split", "Flat" and "CSV", every one of which contains the type of files indicated by its name.
-
- 08 Jan, 2019 4 commits
- 07 Jan, 2019 6 commits
-
-
Serbaf authored
-
Serbaf authored
The possibility of discarding or keeping the intermediate files (split and/or flattened JSONs) has been added as an invocation argument (-k / --keep-files). Before this version these files were always kept. Now they are discarded by default and the -k options permits to keep them. Before this version, the array compression and remove dollar modes didn't apply when there wasn't an include filter (that is, either with exclude filter or without any filters at all). This has been fixed and now any combination of filters and modes should work correctly.
-
Serbaf authored
-
Serbaf authored
-
Serbaf authored
-
Serbaf authored
Before this version, the array compression and remove dollar modes didn't apply when there wasn't an include filter (that is, either with exclude filter or without any filters at all). This has been fixed and now any combination of filters and modes should work correctly.
-
- 04 Jan, 2019 9 commits
- 02 Jan, 2019 5 commits
-
-
Serbaf authored
-
Serbaf authored
Now the user will be prompted whenever a collection with the same path than an already existing one is going to be generated. The user will be able to choose between overwriting or aborting the execution.
-
Serbaf authored
-
Serbaf authored
Filters are no longer hardcoded. Now, if user doesn't specify anything about filters, none will be applied. The user can specify the path to an exclude or an include filter using the -e/--exclude and the -i/--include command-line options. Furthermore, the user can specify -f/--default-filters to use the default filters included in data/filters
-
Serbaf authored
-