Python Pandas - Basics of MultiIndex

---

---

MultiIndex is also called Hierarchical Indexing, it is a powerful feature in pandas that allows you to work with higher-dimensional data in lower-dimensional structures like Series (1D) and DataFrame (2D). With MultiIndex, pandas objects have multiple levels of index labels. Using MultiIndex, you can represent and manipulate data with multiple levels of indexing, making it easier to handle complex data sets efficiently.

In this tutorial, we will learn about the basics of MultiIndex, including how to create MultiIndexed Series and DataFrames, perform basic indexing on MultiIndex axes, and align data using MultiIndex.

Creating MultiIndexed Pandas Objects

There are several ways to create a MultiIndex object in pandas, including from lists of arrays, tuples, products of iterables, or directly from a DataFrame.

Following are the list of helper methods to construct a new MultiIndex −

• MultiIndex.from_arrays()

• MultiIndex.from_product()

• MultiIndex.from_tuples()

• MultiIndex.from_frame()

Creating MultiIndex from Lists of Arrays

By using the pandas.MultiIndex.from_arrays() method we can create MultiIndex from list of arrays.

Example: Creating MultiIndexed Series from List of lists

The following example demonstrates the creation of MultiIndexed Series object using the pandas.MultiIndex.from_arrays() method.

import pandas as pd
import numpy as np

# Create a 2D list
list_2d = [["BMW", "BMW", "Lexus", "Lexus", "foo", "foo", "Audi", "Audi"],
["1", "2", "1", "2", "1", "2", "1", "2"]]

# Create a MultiIndex object
index = pd.MultiIndex.from_arrays(list_2d, names=["first", "second"])

# Creating a MultiIndexed Series
s = pd.Series(np.random.randn(8), index=index)

# Display the output Series 
print("Output MultiIndexed Series:\n",s)

Output

Following is the output of the above code −

Output MultiIndexed Series:
 first  second
BMW    1         0.507702
       2         0.315580
Lexus  1        -0.913939
       2        -0.470642
foo    1        -0.419916
       2        -0.617791
Audi   1        -0.394219
       2         0.324891
dtype: float64

Creating MultiIndex from Tuples

Pandas MultiIndex.from_tuples() method is used to convert list of tuples to MultiIndex.

Example: Creating MultiIndexed DataFrame from Tuples

This example demonstrates the creation of MultiIndexed DataFrame object using the pandas.MultiIndex.from_tuples() method.

import pandas as pd
import numpy as np

# Create a 2D list
list_2d = [["BMW", "BMW", "Lexus", "Lexus", "foo", "foo", "Audi", "Audi"],
["1", "2", "1", "2", "1", "2", "1", "2"]]

# Create a MultiIndex object
tuples = list(zip(*list_2d ))
index = pd.MultiIndex.from_tuples(tuples, names=["first", "second"])

# Creating a MultiIndexed DataFrame
df = pd.DataFrame(np.random.randn(8, 4), index=index, columns=["A", "B", "C", "D"])

# Display the output Series 
print("Output MultiIndexed DataFrame:\n", df)

Output

Following is the output of the above code −

Output MultiIndexed DataFrame:
                      A         B         C         D
first second                                        
BMW   1      -0.936446 -0.274192  0.308845  0.825323
      2       0.418242  0.830447 -0.539598  1.080315
Lexus 1      -1.139546 -1.018409 -1.849736  0.166109
      2      -0.704976  0.503610 -0.689764  1.412166
foo   1       0.464476  1.714391  0.070771 -0.587529
      2       0.427123  1.002659 -0.408395 -1.933066
Audi  1      -0.093110 -0.020240 -0.569153  0.116810
      2      -0.426054 -0.537713 -1.526764 -0.137989

Creating MultiIndex Using from_product()

The Pandas MultiIndex.from_product() method is uses the cartesian product of multiple iterables to create MultiIndex. It is useful when you want every possible combination of elements from two or more iterables.

Example: Creating MultiIndexed DataFrame Using from_product()

This example demonstrates how to create the MultiIndexed DataFrame using the pandas MultiIndex.from_product() method.

import pandas as pd
import numpy as np

# Create a list of lits 
iterable = [[1, 2, 3], ['green', 'black']]

# Create a MultiIndex object
index = pd.MultiIndex.from_product(iterable, names=["number", "color"])

# Creating a MultiIndexed DataFrame
df = pd.DataFrame(np.random.randn(6, 3), index=index, columns=["A", "B", "C"])

# Display the output Series 
print("Output MultiIndexed DataFrame:\n", df)

Output

Following is the output of the above code −

Output MultiIndexed DataFrame:
                      A         B         C
number color                              
1      green  1.399149 -0.995173  1.537441
       black -0.962953 -0.398537  0.072796
2      green  0.064748 -0.148591  0.111019
       black -0.204023 -1.706223  1.415122
3      green -0.219234 -0.113010  0.626351
       black  0.069791 -0.665270  0.900951

Creating MultiIndex from DataFrame

The Pandas MultiIndex.from_frame() method is used to create a MultiIndex from a DataFrame.

Example: Creating MultiIndex from DataFrame

This example uses the pd.MultiIndex.from_frame() method to directly create a MultiIndex object from a DataFrame.

import pandas as pd
import numpy as np

# Create a DataFrame
df = pd.DataFrame([["BMW", 1], ["BMW", 2], ["Lexus", 1],["Lexus", 2]], 
 columns=["first", "second"])

# Create a MultiIndex object
index = pd.MultiIndex.from_frame(df)

# Creating a MultiIndexed DataFrame
df = pd.DataFrame(np.random.randn(4, 3), index=index, columns=["A", "B", "C"])
# Display the output Series 
print("Output MultiIndexed DataFrame:\n", df)

Output

Following is the output of the above code −

Output MultiIndexed DataFrame:
                      A         B         C
first second                              
BMW   1       0.918728 -1.224909 -1.486071
      2      -0.413480 -0.239801  0.000995
Lexus 1       2.550773  0.885128  1.252554
      2       1.077487 -1.021780 -0.360193

Basic Indexing on Axis with MultiIndex

Indexing with MultiIndex used to slice and select data in more flexible ways compared to a regular index.

Example: Selecting Data by Index Level

Here is a basic example demonstrating the indexing MultiIndexed Series object using the .loc[] method.

import pandas as pd
import numpy as np

# Creating MultiIndex from arrays
arrays = [["bar", "bar", "baz", "baz", "foo", "foo", "qux", "qux"],
["one", "two", "one", "two", "one", "two", "one", "two"]]

# Creating a list of tuples from the arrays
tuples = list(zip(*arrays))

# Creating a MultiIndex from tuples
index = pd.MultiIndex.from_tuples(tuples, names=["first", "second"])

# Creating a Series with MultiIndex
s = pd.Series([2, 3, 1, 4, 6, 1, 7, 8], index=index)

print("MultiIndexed Series:\n", s)

# Indexing the MultiIndexed Series using .loc[]
print("\nSelecting data at index ('bar', 'one') and column 'A':")
print(s.loc[('bar', 'one')])

Output

Following is the output of the above code −

MultiIndexed Series:
 first  second
bar    one       2
       two       3
baz    one       1
       two       4
foo    one       6
       two       1
qux    one       7
       two       8
dtype: int64

Selecting data at index ('bar', 'one') and column 'A':
2