Permute list of lists with mixed elements (np.random.permutation() fails with ValueError)

Question

I'm trying to permute a list composed of sublists with mixed-type elements:

import numpy as np

a0 = ['122', 877.503017, 955.471176, [21.701201, 1.315585]]
a1 = ['176', 1134.076908, 1125.504758, [19.436181, 0.9987899]]
a2 = ['177', 1038.686843, 1018.987868, [19.539959, 1.183997]]
a3 = ['178', 878.999081, 1022.050447, [19.6448771, 1.1867719]]

a = [a0, a1, a2, a3]

b = np.random.permutation(a)

This will fail with:

ValueError: cannot set an array element with a sequence

Is there a built in function that will allow me to generate such permutation?

I need to generate a single random permutation, I'm not trying to obtain all the possible permutations.


I checked the three answers given with:

import time
import random

# np.random.permutation()
start = time.time()
for _ in np.arange(100000):
    b = np.random.permutation([np.array(i, dtype='object') for i in a])
print(time.time() - start)

# np.random.shuffle()
start = time.time()
for _ in np.arange(100000):
    b = a[:]
    np.random.shuffle(b)
print(time.time() - start)

# random.shuffle()
start = time.time()
for _ in np.arange(100000):
    random.shuffle(a)
print(time.time() - start)

The results are:

1.47580695152
0.11471414566
0.26300907135

so the np.random.shuffle() solution is about 10x faster than np.random.permutation() and 2x faster than random.shuffle().


Show source
| numpy   | python   | permutation   2016-09-08 21:09 4 Answers

Answers ( 4 )

  1. 2016-09-08 21:09

    You need to convert your list to numpy arrays with with type object(), so that random.permutation() can interpret the lists as numpy types rather than sequence:

    >>> a = [np.array(i, dtype='object') for i in a]
    >>> 
    >>> np.random.permutation(a)
    array([['122', 877.503017, 955.471176, [21.701201, 1.315585]],
           ['177', 1038.686843, 1018.987868, [19.539959, 1.183997]],
           ['178', 878.999081, 1022.050447, [19.6448771, 1.1867719]],
           ['176', 1134.076908, 1125.504758, [19.436181, 0.9987899]]], dtype=object)
    

    You can also use create a uniqe array from your lists using numpy.array() instead of using a list comprehension:

    >>> a = np.array((a0, a1, a2, a3), dtype='object')
    >>> a
    array([['122', 877.503017, 955.471176, [21.701201, 1.315585]],
           ['176', 1134.076908, 1125.504758, [19.436181, 0.9987899]],
           ['177', 1038.686843, 1018.987868, [19.539959, 1.183997]],
           ['178', 878.999081, 1022.050447, [19.6448771, 1.1867719]]], dtype=object)
    >>> np.random.permutation(a)
    array([['122', 877.503017, 955.471176, [21.701201, 1.315585]],
           ['177', 1038.686843, 1018.987868, [19.539959, 1.183997]],
           ['176', 1134.076908, 1125.504758, [19.436181, 0.9987899]],
           ['178', 878.999081, 1022.050447, [19.6448771, 1.1867719]]], dtype=object)
    >>> np.random.permutation(a)
    array([['177', 1038.686843, 1018.987868, [19.539959, 1.183997]],
           ['176', 1134.076908, 1125.504758, [19.436181, 0.9987899]],
           ['178', 878.999081, 1022.050447, [19.6448771, 1.1867719]],
           ['122', 877.503017, 955.471176, [21.701201, 1.315585]]], dtype=object)
    
  2. 2016-09-08 21:09

    What about using np.random.shuffle?

    # if you want the result in another list, otherwise just apply shuffle to a
    b = a[:]
    # shuffle the elements
    np.random.shuffle(b)
    # see the result of the shuffling
    print(b)
    

    See this answer for the difference between shuffle and permutation

  3. 2016-09-08 21:09

    random.shuffle() changes the list in place.

    Python API methods that alter a structure in-place generally return None.

    Please try random.sample(a,len(a))

    The code would look like:

    a = a[:]
    b = random.sample(a,len(a))
    
  4. 2016-09-08 21:09

    If you just want to create a random permutation of a = [a0, a1, a2, a3], might I suggest permuting the indices instead?

    >>> random_indices = np.random.permutation(np.arange(len(a)))
    >>> a_perm = [a[i] for i in random_indices]
    ... # Or just use the indices as you see fit...
    

    If you're using numpy just for this, skip numpy altogether instead and just use random.shuffle to effect the same:

    >>> import random
    >>> random.shuffle(a)
    
◀ Go back