1. 19 Feb, 2019 2 commits
  2. 14 Jan, 2019 8 commits
    • Merge branch 'develop' · de853041
      Serbaf authored
    • Merge branch 'bugfix/arrayquoting' into develop · 2d4876db
      Serbaf authored
    • enhancement of 1.0.1 · 029567ec
      Serbaf authored
    • enhancement of 1.0.1 · 49471344
      Serbaf authored
    • Merge branch 'develop' · 3914bb70
      Serbaf authored
    • Bugfix (1.0.1) · 19ccafd9
      Serbaf authored
      Fixed the array representation. Before this fix, the arrays of values were
      represented as "[a, b, c]", which could lead to tremendous bugs due to
      ambiguity (the example array could have three values, "a", "b" and "c", or only
      one, "a, b, c", or other combinations). The fixed notation solves that
      ambiguity, because now the arrays are represented in the form: ["a", "b", "c"]
    • Merge branch 'develop' · 41982bf2
      Serbaf authored
    • Added argument to opt compression/-c (0.9.0) · d11cc8bf
      Serbaf authored
      Now the user cannot only specify if he wants to represent the arrays flattened
      or compressed in the CSV file, but also which depth of compression wih an
      integer. A level of 0 would equal no compression, a level 1 equals the old
      compression mode (and the current UPV-compatible mode too), that is, to just
      unnest 1 level of arrays, for example:
      
      house[0].dweller[0]: "Ana"
      house[0].dweller[1]: "Roberto"
      house[1].dweller[0]: "Marina"
      
      Would become:
      house[0].dweller: "[Ana, Roberto]"
      house[1].dweller: "[Marina]"
      
      Now, it is also possible to select a compression of depth 2 or more, which
      would provide the followint output:
      
      house.dweller: "[[Ana, Roberto], [Marina]]"
      
      Some minor bugs fixed and tests adapted to the new feature.
      This is the state of the program desired in the beginning of the development,
      and this version (with possible minor fixes) is the one expected for the 1.0.0
      release.
  3. 10 Jan, 2019 4 commits
    • Merge branch 'develop' · 10bf8b68
      Serbaf authored
      Since the program has reached a pretty stable stage (to the level provided by
      automatic and manual testing done by the developer) and with a
      reasonably good functionality in comparison to its initial goals, the
      changes are dropped to master.
      New release version boys!!
    • minor enhancement in the array detecting regex · 4e548465
      Serbaf authored
    • Merge branch 'feature/change_output' into develop · fd18e636
      Serbaf authored
    • Bugfixes (0.8.1) · a48e4a55
      Serbaf authored
      Fixed some bugfixes related to the changes introduced at 0.8.0. E.g., in 0.8.0
      one could not flatten the split directory of the same collection, because the
      collection was removed before (the overwriting was performed over the top
      directory of the collection, and thus the input for the operation was removed
      before beginning to flatten). Tests have been adapted too
  4. 09 Jan, 2019 2 commits
    • minor bugfix · ce3b3b80
      Serbaf authored
    • Large changes in the directory structure of the output (0.8.0) · 050db524
      Serbaf authored
      This patch includes a big reestructuration of the directory hierarchy produced
      by the program. The three standard folders for output (split\_tweets,
      flattened\_tweets and csv\_converted\_tweets) cease to exist. Now the only
      preexistent folder for output is "output" itself. Everytime a collection is
      split, flattened or converted, a new output folder with the name of that
      collection is created inside "output". These collections can have up to three
      subdirectories called "Split", "Flat" and "CSV", every one of which contains
      the type of files indicated by its name.
  5. 08 Jan, 2019 4 commits
  6. 07 Jan, 2019 6 commits
    • minor bugfixes · 1d6a0816
      Serbaf authored
    • Added arg option to keep/discard the intermediate files (0.7.0) · 97a4902f
      Serbaf authored
      The possibility of discarding or keeping the intermediate files (split and/or
      flattened JSONs) has been added as an invocation argument (-k / --keep-files).
      Before this version these files were always kept. Now they are discarded by
      default and the -k options permits to keep them.
      Before this version, the array compression and remove dollar modes didn't apply
      when there wasn't an include filter (that is, either with exclude filter or
      without any filters at all). This has been fixed and now any combination of
      filters and modes should work correctly.
    • Merge branch 'feature/unittesting2' into develop · e6fcd2d4
      Serbaf authored
    • more and moar unittests · e222c6b0
      Serbaf authored
    • more unittests! · 1a2ba62e
      Serbaf authored
    • Bugfix (0.6.9) · 1bc29ec4
      Serbaf authored
      Before this version, the array compression and remove dollar modes didn't apply
      when there wasn't an include filter (that is, either with exclude filter or
      without any filters at all). This has been fixed and now any combination of
      filters and modes should work correctly.
  7. 04 Jan, 2019 9 commits
  8. 02 Jan, 2019 5 commits
    • Overwrite prompts (0.6.8) · eaff7f7a
      Serbaf authored
      Now the user will be prompted whenever a collection with the same path
      than an already existing one is going to be generated. The user will be
      able to choose between overwriting or aborting the execution.
    • Merge branch 'feature/filters-opt' into develop · 13a4f85b
      Serbaf authored
    • New filtering options (0.6.7) · 0b2695e5
      Serbaf authored
      Filters are no longer hardcoded. Now, if user doesn't specify anything
      about filters, none will be applied. The user can specify the path to an
      exclude or an include filter using the -e/--exclude and the -i/--include
      command-line options. Furthermore, the user can specify
      -f/--default-filters to use the default filters included in data/filters
    • Updated help · a322636f
      Serbaf authored