myprojects-hg-reborn

Fork

A simple script to get SMEs and broadband statistics from the scoreboard.

I introduced a cut-off on the years for the railways aid.

I added an option about including an extra column on TCF/TCTF.

I now add info about the TCTF cases and I treat separately the cases when I create a single file or I keep the tam separate from Poland.

I added command to compress the csv files.

I limit the number of open csv files while writing the data which improved the performance.

I now save more efficiently the csv files.

I cleaned up the code and I again went for a multifile solution.

I went back to a multi-file parquet solution. It uses much less memory.

I now ensure that beneficiary_region is text.

I now read a compressed csv file and I save the data as multiple files. It is much easier on memory.

New file to handle the latest version of the Polish tam.

I improved the fruit repository.

I added new entries of my sources.list.d directory.

I added a file to configure mpv.

I now save the parquet files with a more expressive name.

The solution with write_dataset is better when working with very large files. I write multiple parquet files and I have no ram issues at all.

I added a new code showing how to handle a big csv file, giving it an automatic schema and filter the results.

I now ensure that I remove final_spain.csv before I generate it.

I added another file to carry out some diagnostics on TAM.

I made a mistake with the saving of a compressed csv file.

I modified the obfuscated data set.

I save the data directly as a compressed csv file.

I changed the way in which I handle the date.

I changed the file names and I now use write_csv_arrow to save the data as a csv.

I now can choose to skip the first part of the data processing once it has been done for all.

I now use write_parquet instead of write_dataset to save the results. I am not interested in a multifile solution.

I changed the name of the input file.

I remove two columns I do not need any longer.

I modified the code because of modifications in the Slovenian input files.