I am trying to replace "bad values" below and above of thresholds with a default value (e.g. setting them to NaN).
I am unsing a numpy array with 1000k values and more - so performance is an issue.
My prototype does the operation in two steps, is t... more
I have an array of images that I want to feed to TensorFlow. I want to center the images around the mean and standardize the standard deviation. I followed this answer but I cannot seem to get the mean to zero. I am learning numpy so maybe I am missi... more
I have a 3-dimensional tensor A of size (M,N,N). I also have a weight vector p of length M. I want to compute
The dimension N can be large so I want to implement it in the efficient way possible. I am using the following code:
import numpy as np
I'm using scipy's method integrate.odeint to solve a second order LDE. The method requires that the equation be put in the form of a system of two first-order equations in two unknowns. The method
I've got a 2dim database as a numpy.array (time,field values). Secondly I have a np.array of tuples, containing the coordinates of the field in the same order. What I want to do now is to reshape the database so that I can later view the map with mat... more
I need to solve a large set of linear systems, in the least-squares sense. I am having trouble in understanding the difference in computational efficiency of numpy.linalg.lstsq(a, b), np.dot(np.linalg.pinv(a), b) and the mathematical implementation.
I am getting an optimize warning:
OptimizeWarning: Covariance of the parameters could not be estimated
when trying to fit my piecewise function to my data using scipy.optimize.curve_fit. Meaning no fittin... more
I'm trying to find an efficient way to generate rolling counts or sums in pandas given a grouping and a date range. Eventually, I want to be able to add conditions, ie. evaluating a 'type' field, but I'm not there just yet. I've written something to ... more
I have calculated some ufloats using the uncertainties package, let's call them theta and gamma.
Now, for my following calculations I don't need their errors (I am using them later as y-values in linear plot).
I simply could calculate them ignoring ... more
I'm iterating through a list of words to find the most frequently used character between words (i.e. in list [hello, hank], 'h' counts as appearing twice while 'l' counts as appearing once.). A python list works fine, but I'm also looking into NumPy ... more
I maintain a little Python package that converts between different formats used for mesh representation à la
Those files can grow pretty big, so when reading them with Python it's important to do it efficiently.
One of the most used formats is m... more
a (n, d) dimensional matrix x
a (d, L) dimensional matrix c
Let's denote the b-th row of x with x_b and the i-th column of c with c_i. I want to calculate the (n, L) dimensional matrix diff which contains at (b, i):
|| x_b - c_i ||^2,
Is there an expression (perhaps using np.tensordot) that most expressively captures a sparse matrix vector multplicatiuon between a block matrix of diagonal blocks and a vector of the corresponding size?
I have a working implementation that performs... more
So I'm just trying to write a simple script to convert RBG to YUV and I have ended up with something like this:
rgb2yuv_matrix = np.array([[0.299, 0.587, 0.114], [-0.1473, -0.28886, 0.436],[0.615, -0.51499, 0.10001]])
for i in range(n_train):
I have a numpy array embed_vec of length tot_vec in which each entry is a 3d vector:
[[ 0.52483319 0.78015841 0.71117216]
[ 0.53041481 0.79462171 0.67234534]
[ 0.53645428 0.80896727 0.63119403]
[ 0.72283509 0.40070804 0.15220522]
In the python library pytides, I came across a strange method to initialize a class (the Tide class and its initialization ).
I reproduced below a simplified version of the code :
import numpy as np
I am working with last.fm dataset from the Million song dataset.
The data is available as a set of json-encoded text files where the keys are: track_id, artist, title, timestamp, similars and tags.
Using the similars and track_id fields, I'm trying ... more
I am trying to focus on a Region of Interest on the face by using Numpy cropping. For large images, I have no trouble. But for smaller ones, I can find the face and find the matching bounding box, but when I try to crop it and show the image, I get a... more
I have a 2d numpy array with a small range of different values. I want to retrieve a random index, but with a specific value. For example say this is the array:
arr = [[1,2,1,1,3],
I have two dataframes:
A = pd.DataFrame(data=np.array([['t1',1,'t2',2]]).reshape(2,2),columns=['a','b'])
0 t1 1
1 t2 2
B = pd.DataFrame(data=np.array([[1,2,3],[2,5,6],[3,6,7]]).reshape(3,3),columns=['x','y','z'])
A pandas dataframe column series, same_group needs to be created from booleans according to the values of two existing columns, row and col. The row needs to show True if both cells across a row have similar values (intersecting ... more
I am running the code below but it's giving me an error about arrays. I have tried to find a solution and somehow understand the problem but I couldn't solve the problem. Here is my code:
import tensorflow as tf
import pandas as pa
import numpy as n... more
I have the following list of various numpy arrays:
nparrays_list = [
array([1, 2, 3, 4])
array([5, 6, 7, 8]),
array([9, 10, 11, 12])
I want to iterate through the entire list without affecting the shape of the list (i.e. I don't want... more
A pandas dataframe column series, same_group needs to be created from booleans according to the values of two existing columns, row and col. The row needs to show True if both values have similar values (intersecting values) in a... more
I have such piece of code, where I try to load four columns from csv file
import numpy as np
rtype = np.dtype([('1', np.float), ('2', np.float), ('3', np.float), ('tier', np.str, 32)])
x1, x2, x3, x4 = np.genfromtxt("../Data/out.txt", dtype=rtyp... more
Is there a more pythonic/numpythonic way to do some sort of nested/hierarchical slicing, i.e. a prettier version of this:
_sum = 0
for i in np.arange(n):
_sum += someFunc(A[i,:])
Basically I would like to map someFunc (which takes arrays of any... more
What's the fastest way of converting a 1d Numpy array containing only 0s and 1s to a unique integer?
The best I came up with so far is to use Cython and see the array as a mirrored binary number*.
Let's say I have a numpy array of some integer type (say np.int64) and want to cast it to another type (say np.int8). How can I most effectively check if the operation is safe (preserving all values)?
There are two approaches I've come up with:
I am reading data in chunks from a CSV like this :
for chunk in pd.read_csv(file, chunksize=50000, names = col_names, header = 0, dtype = dtype):
chunk['derived_field_1'] = [1 if x == 'High' else -1 for x in chunk['indicator']]
In order to calculate the image mean, one can use numpy.mean() if I'm not mistaken.
What mean() does, is simply summing all elements and then dividing by the number of all elements summed. furthermore if I want to calculate mean channelwise,(each cha... more
I have a dataframe and I want to return a subset (new copy not reference) of this dataframe to perform some operations. However I find it unable to filter on the criteria i need.
I need these three criteria to filer :
1. df['A'] != NaN
2. df['B']... more
I am actually using pandas.read_csv to read a large (5GB, ~97million rows X 7 columns) txt file with python (a point cloud).
My need is to read the first three columns (which represent x, y, z coordinates), and retrieve the bounding-box of my point ... more
I am trying to create the following type of matrix using scipy or numpy.
A_1 = diag(0.5, 0, 0, ...., 0)
A_k = diag(0,0, ..., 1, 0, 0, ....,0) for each 1 < k < N
A_N = diag(0, 0, ...., 0, 0.5)
Essentially A_ks are a series of a diagonal matr... more
I'm looking for either an existing library function from some Python library or a custom Python function using numpy or pandas that is fast, that does the following: takes as input a list of edges in a bipartite graph and returns the subset of the ed... more