Using .sub() over a variable length dataframe?

I have a need to calculate the difference between one static column in a dataframe and some variable amount of columns that make up the rest of the dataframe. My data look like this: School District | 01-01-2017 | 01-02-2017 |...| 02-01-2017 | Base...
more »

2017-06-21 21:06 (2) Answers

Numpy Array Subtraction

How would I go about comparing two different numpy arrays to create a third array? I want to write a loop that goes through two arrays and prints a new array "c" with only the values that are not in a or b For example say I have a = [1,2,3,4] b =...
more »

2017-06-21 18:06 (1) Answers

Ignoring NaN in a dataframe

I want to find the unique elements in a column of a dataframe which have missing values. i tried this: df[Column_name].unique() but it returns nan as one of the elements. what can i do to just ignore the missing values. dataframe look like this.click...
more »

2017-06-21 18:06 (1) Answers

Creating an ordered matrix through indexing

I have a problem related to a detector that reads out the amount of photons that get fed into channels and the times they enter the detector, Lets say its channels 0 to 6 for simplicity sake. Array A will hold the channels, basically the index list, ...
more »

2017-06-21 05:06 (1) Answers

Python numpy unwrap function

I am hoping to convert a array of radians into range [0, 2*pi) and numpy unwrap function is exactly what I need However, when I run the following code to input a = [pi, 2*pi, 3*pi]: import numpy as np a = np.array([np.pi, 2*np.pi, 3*np.pi]) np....
more »

2017-06-21 00:06 (2) Answers

Using scipy kmeans for cluster analysis

I want to understand scipy.cluster.vq.kmeans. Having a number of points distributed in 2D space, the problem is to group them into clusters. This problem came to my attention reading this question and I was thinking that scipy.cluster.vq.kmeans wou...
more »

2017-06-20 20:06 (1) Answers

numpy dot product with missing values

How do you do a numpy dot product where the two vectors might have missing values? This seems to require many additional steps, is there an easier way to do this?: v1 = np.array([1,4,2,np.nan,3]) v2 = np.array([np.nan,np.nan,2,4,1]) np.where(np.isna...
more »

2017-06-20 19:06 (1) Answers

returning numpy arrays via pybind11

I have a C++ function computing a large tensor which I would like to return to Python as a NumPy array via pybind11. From the documentation of pybind11, it seems like using STL unique_ptr is desirable. In the following example, the commented out ...
more »

2017-06-20 19:06 (1) Answers

Pandas column of lists to separate columns

Problem Incoming data is a list of 0+ categories: #input data frame df = pd.DataFrame({'categories':(list('ABC'), list('BC'), list('A'))}) categories 0 [A, B, C] 1 [B, C] 2 [A] I would like to convert this to a DataFrame with one c...
more »

2017-06-20 17:06 (2) Answers

python numpy array indexing

I think I understand the indexing of array in python/numpy correctly. But today I met a problem as follows: I have a 6-d array e.g. A and A.shape = (11,1,9,1,5,7). Then I use the indexing as follows: B = A[:,0,0,0,[3,4,2],0] and B.shape = (11,3...
more »

2017-06-20 17:06 (1) Answers

How to write DataFrame-friendly functions

Recently I've been making the switch from using numpy's ndarray to pandas' DataFrame for my data analysis needs. I've noticed that numpy functions seem to accept DataFrame objects in place of ndarray without issue. However, when I try to use many of ...
more »

2017-06-20 16:06 (1) Answers

replace values by the mean

I have a dataframe as follow: Col1 Price 1 Plastic 50 2 Metal 100 3 Plastic 40 I would like to replace values in Col1 by their means in Price, So I get: Col1 Price 1 45 ...
more »

2017-06-19 18:06 (3) Answers

Convert 'int' to pytorch 'Variable' makes problems

First project with pytorch and I got stuck trying to convert an MNIST label 'int' into a torch 'Variable'. Debugger says it has no dimension?! # numpy mnist data X_train, Y_train = read_data("training") X_test , Y_test = read_data("testing") arr =...
more »

2017-06-19 15:06 (1) Answers