Get data sequentially from numpy array in Matlab ordering

Question

As an example, suppose, in Matlab, a Matrix a(2,3,2) like this:

a(:,:,1) =

     1     2     3
     4     5     6


a(:,:,2) =

     7     8     9
    10    11    12

If I use mex and access the elements of this matrix sequentially, I get the following order (in the end, a code to access them sequentially):

1, 4, 2, 5, 3, 6, 7, 10, 8, 11, 9, 12

Now, if I have the same matrix in numpy

In [2]: np.array([[[1,2,3],[4,5,6]],[[7,8,9],[10,11,12]]])
Out[2]: 
array([[[ 1,  2,  3],
        [ 4,  5,  6]],

       [[ 7,  8,  9],
        [10, 11, 12]]])

, I could try to use .flatten(...) to access the items sequentially (couldn't find a better way to do it -- I am open to suggestions). flatten() has 4 "ordering" options:

In [4]: a.flatten('F')
Out[4]: array([ 1,  7,  4, 10,  2,  8,  5, 11,  3,  9,  6, 12])

In [5]: a.flatten('C')
Out[5]: array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12])

In [6]: a.flatten('A')
Out[6]: array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12])

In [7]: a.flatten('K')
Out[7]: array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12])

Is there an elegant way to access the elements of the numpy array in a Matlab ordering? (in my real use case, these matrix are big, so copiyng them around wouldn't be preferred)

Appendix: Code to print the matrix sequentially

[not great, I know, just for testing]

  1 void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
  2 {
  3         // This just a test: hardcoding the size of `data_input`
  4         int data_input_size = 12;
  5         double *data_input;
  6         const mxArray *batch_data;
  7 
  8         // Gets a pointer to the first element of batch_data
  9         data_input  = mxGetPr(prhs[0]);
 10 
 11         for(int i = 0; i < data_input_size; i++) {
 12                 printf("working_data[i]: %f\n", data_input[i]);
 13         }
 14         plhs[0] = mxCreateDoubleMatrix(0, 0, mxREAL);
 15 }

Show source
| numpy   | python   | matlab   2017-01-06 15:01 2 Answers

Answers to Get data sequentially from numpy array in Matlab ordering ( 2 )

  1. 2017-01-06 15:01

    This bugs a lot of people going into NumPy/Python from MATLAB. So, in MATLAB, the indexing format is (column x row x dim3) and so on. With NumPy, it's (axis-0, axis-1, axis-2) and so on.

    To show this schematically using a sample case on MATLAB :

    >> a = reshape(1:27,[3,3,3]);
    >> a
    a(:,:,1) =
    
            row
        --------------->
         1     4     7   |         |
         2     5     8   | col     |
         3     6     9   v         |
    a(:,:,2) =                     |
        10    13    16             | dim3
        11    14    17             |
        12    15    18             |
    a(:,:,3) =                     |
        19    22    25             |
        20    23    26             |
        21    24    27             v
    

    On NumPy :

    In [62]: a = np.arange(27).reshape(3,3,3)
    
    In [63]: a
    Out[63]: 
    
                axis=2
             ---------->
    array([[[ 0,  1,  2],   |          |
            [ 3,  4,  5],   | axis=1   |
            [ 6,  7,  8]],  v          |
                                       |
           [[ 9, 10, 11],              |
            [12, 13, 14],              | axis=0
            [15, 16, 17]],             |
                                       |
           [[18, 19, 20],              |
            [21, 22, 23],              |
            [24, 25, 26]]])            v
    

    Let's try to correlate these dimensions and axes terminology for the 3D array case listed in the question between these two environments :

    MATLAB      NumPy
    ------------------
    cols        axis-1
    rows        axis-2
    dim3        axis-0
    

    Thus, to simulate the same behavior in NumPy as MATLAB, we need the axes in NumPy as : (1,2,0). Together with NumPy's way of storing elements starting from the last axis to the first one i.e. in reversed order, the required axes order would be (0,2,1).

    To perform the permuting of axes that way, we could use np.transpose and thereafter use a flattening operation with np.ravel() -

    a.transpose(0,2,1).ravel()
    

    Sample run -

    In [515]: a
    Out[515]: 
    array([[[ 1,  2,  3],
            [ 4,  5,  6]],
    
           [[ 7,  8,  9],
            [10, 11, 12]]])
    
    In [516]: a.transpose(0,2,1) # Permute axes
    Out[516]: 
    array([[[ 1,  4],
            [ 2,  5],
            [ 3,  6]],
    
           [[ 7, 10],
            [ 8, 11],
            [ 9, 12]]])
    
    In [517]: a.transpose(0,2,1).ravel() # Flattened array
    Out[517]: array([ 1,  4,  2,  5,  3,  6,  7, 10,  8, 11,  9, 12])
    
  2. 2017-01-06 19:01

    This is intended as a suplement to @Divakar's answer.

    In Octave I made x and a and saved them to a file

    >> x=[1, 4, 2, 5, 3, 6, 7, 10, 8, 11, 9, 12]
    >> a=reshape(x,2,3,2)
    
    a =
    
    ans(:,:,1) =
    
       1   2   3
       4   5   6
    
    ans(:,:,2) =
    
        7    8    9
       10   11   12
    

    I can flatten a with (same as your mex):

    >> a(:).'
    

    In numpy, with loadmat from scipy.io

    In [564]: data = loadmat('test.mat')
    In [566]: data['x']
    Out[566]: 
    array([[  1.,   4.,   2.,   5.,   3.,   6.,   7.,  10.,   8.,  11.,   9.,
             12.]])
    In [567]: data['a']
    Out[567]: 
    array([[[  1.,   7.],
            [  2.,   8.],
            [  3.,   9.]],
    
           [[  4.,  10.],
            [  5.,  11.],
            [  6.,  12.]]])
    

    ravel with order 'F' displays the original flat x:

    In [571]: data['a'].ravel(order='F')
    Out[571]: 
    array([  1.,   4.,   2.,   5.,   3.,   6.,   7.,  10.,   8.,  11.,   9., 12.])
    

    To help keep track of shapes I prefer to make them all different. For example

    >> x=0:23;
    >> a=reshape(x,2,3,4);
    

    and a displays as

    ans(:,:,1) =
    
       0   2   4
       1   3   5
    ...
    ans(:,:,4) =
    
       18   20   22
       19   21   23
    

    It is showing 4 (2,3) matrices.

    On the numpy side I get:

    In [575]: data['a'].shape
    Out[575]: (2, 3, 4)
    In [577]: data['a']
    Out[577]: 
    array([[[  0.,   6.,  12.,  18.],
            [  2.,   8.,  14.,  20.],
            [  4.,  10.,  16.,  22.]],
    
           [[  1.,   7.,  13.,  19.],
            [  3.,   9.,  15.,  21.],
            [  5.,  11.,  17.,  23.]]])
    

    This is 2 blocks of (3,4) shape.

    In [579]: data['a'].ravel('F')
    Out[579]: 
    array([  0.,   1.,   2.,   3.,   4.,   5.,   ...,  23.])
    

    I see two key differences between numpy and MATLAB/Octave. Default order is different, but numpy lets you specify F. And display of 3d (and larger) arrays is different. There's in effect a different notion of 'planes' or blocks.

    np.transpose changes the shape, strides, and order of an array, while keeping the same data base.

    Applied to a, it produces a (4,3,2) array with order 'C':

    In [580]: data['a'].T
    Out[580]: 
    array([[[  0.,   1.],
            [  2.,   3.],
            [  4.,   5.]],
    
     ....
           [[ 18.,  19.],
            [ 20.,  21.],
            [ 22.,  23.]]])
    In [581]: data['a'].T.ravel()
    Out[581]: 
    array([  0.,   1.,   2.,   3., ...22.,  23.])
    

    The MATLAB display can be replicated with:

    In [584]: for i in range(4):
         ...:     print(data['a'][:,:,i])
         ...:     
    [[ 0.  2.  4.]
     [ 1.  3.  5.]]
    [[  6.   8.  10.]
     [  7.   9.  11.]]
    [[ 12.  14.  16.]
     [ 13.  15.  17.]]
    [[ 18.  20.  22.]
     [ 19.  21.  23.]]
    

Leave a reply to - Get data sequentially from numpy array in Matlab ordering

◀ Go back