CSV Import/Export, best practice regarding locale

I am exporting and importing data from my Android app as .csv I had to make several decisions, and I wonder how much trouble it will generate in the field. I separate the columns using ";" and I export strings, integers, floats and date/time as fol...
more »

2017-08-12 19:08 (0) Answers

Pandas read csv is shifting columns

I'm trying to create a dataframe of a csv file that has 4 empty columns. When I open it on LibreOffice or Excel it correctly identifies the empty columns. However, opening with DataFrame.read_csv() end up shifting the columns' values by one. How ca...
more »

2017-08-12 18:08 (1) Answers

Pandas `read_csv` Method Is Using Too Much RAM

I am currently playing with Rotten Tomatoes dataset on Kaggle while using pandas DataFrame() to manipulate the data. I have implemented CountVectorizer() from sklearn to extract the features (the size is 5000). I have then saved 100k rows of featur...
more »

2017-08-06 15:08 (1) Answers

Merging multi row csv file into one using python

I have the following dictionary: details = {"Primary_key" : [{'key_1': 'val', 'key_3': "val", 'key_5': ['val_1', 'val_2', 'val_3'], 'key_6': 'val'}, {'key_2': 'val', 'key_3': 'val', 'key_5': ['val_1','val_2'], 'key_6': 'val'}, {'key_1': 'val', 'key...
more »

2017-08-06 05:08 (2) Answers

Values can't be uploaded to database

I am trying to upload the values of the csv file to database through the fallowing code <?php include('../connection.php'); $class = $_POST['class']; echo "hello"; if(isset($_FILES['filename'])){ echo "hello"; $filename = $_FILES['fil...
more »

2017-08-05 15:08 (1) Answers

sqlloader with constraint NOT NULL in oracle table

I'm entering values from a CSV file to an Oracle table using a SQL*Loader script. In this table there are fields with NOT NULL constraints. In my CSV file the corresponding field is "" and I would like put a blank string into the Oracle table when t...
more »

2017-08-04 18:08 (1) Answers

write.csv precision R

I am dealing with very precise numbers (maximum number of digits). I noticed that write.csv(x) in R sometimes round the number. Has anyone noticed something like that? What is the default number of digits saved? ...
more »

2017-08-04 09:08 (1) Answers

How to call functions from another php file?

In short: I am uploading to /var/tmp multiple excel files, then convert them into .csv(2 different converters for .xls and .xlsx). The resulting file, result.csv should be inserted into database. It all worked until we decided to allow to upload mult...
more »

2017-07-26 12:07 (3) Answers

CakePHP 3 - CSV exporter according to user inputs

Hi i'm working with a CSV exporter and it works pretty well for me. All credits to this guy (https://github.com/FriendsOfCake/cakephp-csvview) who did an amazing job at this. So by default if i want to get some records displayed on a table to a csv...
more »

2017-07-24 09:07 (1) Answers

Keep rows with max value of a specific column

I am new to Python and I want to do the following. I have a csv file (input.csv) which contains a header row and 4 columns. A part of this csv file is depicted below: gene-name p-value stepup(p-value) fold-change IFIT1 6.79175E-005 0.0874312 96.0464...
more »

2017-07-21 19:07 (2) Answers

String.replace and same method call mutliple times

I and my friend has written below code to prevent code injection in injection in CSV files. (written in java) For big CSV files (say 400 columns, 10000 rows), the code is taking ~15 seconds in the worst case (all columns and rows are bad). Can anybo...
more »

2017-07-21 13:07 (2) Answers

Downloading files from URL in Python

I have a CSV file that has 67000 rows of URL. Each URL leads to download other datasets in format of CSV, HWP, ZIP, etc. This is the code I have written: import cgi import requests SAVE_DIR = 'C:/dataset' def downloadURLResource(url): r = re...
more »

2017-07-20 10:07 (1) Answers

Print JSON data from csv list of multiple urls

Very new to Python and haven't found specific answer on SO but apologies in advance if this appears very naive or elsewhere already. I am trying to print 'IncorporationDate' JSON data from multiple urls of public data set. I have the urls saved as a...
more »

2017-07-06 17:07 (2) Answers

Reading CSV in Reverse Header order

My api provides CSV in pre-formatted form. Suppose it has a format of : id country city state But when I read the csv it will return data in same order as per header, which does not provide the correct value for combined Address. Whereas I want...
more »

2017-07-06 16:07 (0) Answers