Load Data File and Initial EDA

In [1]:

# expand border width
from IPython.display import display, HTML
display(HTML("<style>.container { width:100% !important; }</style>"))

# Needed libraries
# Data manipulation
import pandas as pd
import numpy as np


# Data Visualization
import matplotlib.pyplot as plt
import seaborn as sns

%matplotlib inline

# hide warning messages
import warnings
warnings.filterwarnings('ignore')

pd.set_option('display.max_columns', None)  # Show all columns
pd.set_option('display.width', 150)  # Adjust the width as needed

Set location of the data file and read it into a dataframe

In [2]:

#from google.colab import drive
#drive.mount('/content/drive')
dataFileNamePath = "./data/data.csv"

In [3]:

masterRawData = pd.read_csv(dataFileNamePath)
print("The dataframe shape is: ", masterRawData.shape, ": ", masterRawData.shape[0], "- Rows, ", masterRawData.shape[1], "- Columns\n")
masterRawData.head()

Out [3]:

The dataframe shape is:  (4424, 37) :  4424 - Rows,  37 - Columns

	Marital status	Application mode	Application order	Course	Daytime/evening attendance\t	Previous qualification	Previous qualification (grade)	Nacionality	Mother's qualification	Father's qualification	Mother's occupation	Father's occupation	Admission grade	Displaced	Tuition fees up to date	Gender	Age at enrollment	Curricular units 1st sem (enrolled)	Curricular units 1st sem (evaluations)	Curricular units 1st sem (approved)	Curricular units 1st sem (grade)	Curricular units 2nd sem (enrolled)	Curricular units 2nd sem (evaluations)	Curricular units 2nd sem (approved)	Curricular units 2nd sem (grade)	Unemployment rate	Inflation rate	GDP	Target
0	1	17	5	171	1	1	122.0	1	19	12	5	9	127.3	1	1	1	20	0	0	0	0.000000	0	0	0	0.000000	10.8	1.4	1.74	Dropout
1	1	15	1	9254	1	1	160.0	1	1	3	3	3	142.5	1	0	1	19	6	6	6	14.000000	6	6	6	13.666667	13.9	-0.3	0.79	Graduate
2	1	1	5	9070	1	1	122.0	1	37	37	9	9	124.8	1	0	1	19	6	0	0	0.000000	6	0	0	0.000000	10.8	1.4	1.74	Dropout
3	1	17	2	9773	1	1	122.0	1	38	37	5	3	119.6	1	1	0	20	6	8	6	13.428571	6	10	5	12.400000	9.4	-0.8	-3.12	Graduate
4	2	39	1	8014	0	1	100.0	1	37	38	9	9	141.5	0	1	0	45	6	9	5	12.333333	6	6	6	13.000000	13.9	-0.3	0.79	Graduate

There are 37 features and 4424 instances

In [4]:

data = masterRawData.copy()

In [5]:

print("Head\n", data.head())  # prefer to use print here with display width adjusted to force wrapping text to view all columns at once and avoid having to scroll left to right
print("\nTail\n", data.tail())

Head
    Marital status  Application mode  Application order  Course  Daytime/evening attendance\t  Previous qualification  \
0               1                17                  5     171                             1                       1   
1               1                15                  1    9254                             1                       1   
2               1                 1                  5    9070                             1                       1   
3               1                17                  2    9773                             1                       1   
4               2                39                  1    8014                             0                       1   

   Previous qualification (grade)  Nacionality  Mother's qualification  Father's qualification  Mother's occupation  Father's occupation  \
0                           122.0            1                      19                      12                    5                    9   
1                           160.0            1                       1                       3                    3                    3   
2                           122.0            1                      37                      37                    9                    9   
3                           122.0            1                      38                      37                    5                    3   
4                           100.0            1                      37                      38                    9                    9   

   Admission grade  Displaced  Educational special needs  Debtor  Tuition fees up to date  Gender  Scholarship holder  Age at enrollment  \
0            127.3          1                          0       0                        1       1                   0                 20   
1            142.5          1                          0       0                        0       1                   0                 19   
2            124.8          1                          0       0                        0       1                   0                 19   
3            119.6          1                          0       0                        1       0                   0                 20   
4            141.5          0                          0       0                        1       0                   0                 45   

   International  Curricular units 1st sem (credited)  Curricular units 1st sem (enrolled)  Curricular units 1st sem (evaluations)  \
0              0                                    0                                    0                                       0   
1              0                                    0                                    6                                       6   
2              0                                    0                                    6                                       0   
3              0                                    0                                    6                                       8   
4              0                                    0                                    6                                       9   

   Curricular units 1st sem (approved)  Curricular units 1st sem (grade)  Curricular units 1st sem (without evaluations)  \
0                                    0                          0.000000                                               0   
1                                    6                         14.000000                                               0   
2                                    0                          0.000000                                               0   
3                                    6                         13.428571                                               0   
4                                    5                         12.333333                                               0   

   Curricular units 2nd sem (credited)  Curricular units 2nd sem (enrolled)  Curricular units 2nd sem (evaluations)  \
0                                    0                                    0                                       0   
1                                    0                                    6                                       6   
2                                    0                                    6                                       0   
3                                    0                                    6                                      10   
4                                    0                                    6                                       6   

   Curricular units 2nd sem (approved)  Curricular units 2nd sem (grade)  Curricular units 2nd sem (without evaluations)  Unemployment rate  \
0                                    0                          0.000000                                               0               10.8   
1                                    6                         13.666667                                               0               13.9   
2                                    0                          0.000000                                               0               10.8   
3                                    5                         12.400000                                               0                9.4   
4                                    6                         13.000000                                               0               13.9   

   Inflation rate   GDP    Target  
0             1.4  1.74   Dropout  
1            -0.3  0.79  Graduate  
2             1.4  1.74   Dropout  
3            -0.8 -3.12  Graduate  
4            -0.3  0.79  Graduate  

Tail
       Marital status  Application mode  Application order  Course  Daytime/evening attendance\t  Previous qualification  \
4419               1                 1                  6    9773                             1                       1   
4420               1                 1                  2    9773                             1                       1   
4421               1                 1                  1    9500                             1                       1   
4422               1                 1                  1    9147                             1                       1   
4423               1                10                  1    9773                             1                       1   

      Previous qualification (grade)  Nacionality  Mother's qualification  Father's qualification  Mother's occupation  Father's occupation  \
4419                           125.0            1                       1                       1                    5                    4   
4420                           120.0          105                       1                       1                    9                    9   
4421                           154.0            1                      37                      37                    9                    9   
4422                           180.0            1                      37                      37                    7                    4   
4423                           152.0           22                      38                      37                    5                    9   

      Admission grade  Displaced  Educational special needs  Debtor  Tuition fees up to date  Gender  Scholarship holder  Age at enrollment  \
4419            122.2          0                          0       0                        1       1                   0                 19   
4420            119.0          1                          0       1                        0       0                   0                 18   
4421            149.5          1                          0       0                        1       0                   1                 30   
4422            153.8          1                          0       0                        1       0                   1                 20   
4423            152.0          1                          0       0                        1       0                   0                 22   

      International  Curricular units 1st sem (credited)  Curricular units 1st sem (enrolled)  Curricular units 1st sem (evaluations)  \
4419              0                                    0                                    6                                       7   
4420              1                                    0                                    6                                       6   
4421              0                                    0                                    7                                       8   
4422              0                                    0                                    5                                       5   
4423              1                                    0                                    6                                       8   

      Curricular units 1st sem (approved)  Curricular units 1st sem (grade)  Curricular units 1st sem (without evaluations)  \
4419                                    5                         13.600000                                               0   
4420                                    6                         12.000000                                               0   
4421                                    7                         14.912500                                               0   
4422                                    5                         13.800000                                               0   
4423                                    6                         11.666667                                               0   

      Curricular units 2nd sem (credited)  Curricular units 2nd sem (enrolled)  Curricular units 2nd sem (evaluations)  \
4419                                    0                                    6                                       8   
4420                                    0                                    6                                       6   
4421                                    0                                    8                                       9   
4422                                    0                                    5                                       6   
4423                                    0                                    6                                       6   

      Curricular units 2nd sem (approved)  Curricular units 2nd sem (grade)  Curricular units 2nd sem (without evaluations)  Unemployment rate  \
4419                                    5                         12.666667                                               0               15.5   
4420                                    2                         11.000000                                               0               11.1   
4421                                    1                         13.500000                                               0               13.9   
4422                                    5                         12.000000                                               0                9.4   
4423                                    6                         13.000000                                               0               12.7   

      Inflation rate   GDP    Target  
4419             2.8 -4.06  Graduate  
4420             0.6  2.02   Dropout  
4421            -0.3  0.79   Dropout  
4422            -0.8 -3.12  Graduate  
4423             3.7 -1.70  Graduate  
      Marital status  Application mode  Application order  Course  Daytime/evening attendance\t  Previous qualification  \
4419               1                 1                  6    9773                             1                       1   
4420               1                 1                  2    9773                             1                       1   
4421               1                 1                  1    9500                             1                       1   
4422               1                 1                  1    9147                             1                       1   
4423               1                10                  1    9773                             1                       1   

      Previous qualification (grade)  Nacionality  Mother's qualification  Father's qualification  Mother's occupation  Father's occupation  \
4419                           125.0            1                       1                       1                    5                    4   
4420                           120.0          105                       1                       1                    9                    9   
4421                           154.0            1                      37                      37                    9                    9   
4422                           180.0            1                      37                      37                    7                    4   
4423                           152.0           22                      38                      37                    5                    9   

      Admission grade  Displaced  Educational special needs  Debtor  Tuition fees up to date  Gender  Scholarship holder  Age at enrollment  \
4419            122.2          0                          0       0                        1       1                   0                 19   
4420            119.0          1                          0       1                        0       0                   0                 18   
4421            149.5          1                          0       0                        1       0                   1                 30   
4422            153.8          1                          0       0                        1       0                   1                 20   
4423            152.0          1                          0       0                        1       0                   0                 22   

      International  Curricular units 1st sem (credited)  Curricular units 1st sem (enrolled)  Curricular units 1st sem (evaluations)  \
4419              0                                    0                                    6                                       7   
4420              1                                    0                                    6                                       6   
4421              0                                    0                                    7                                       8   
4422              0                                    0                                    5                                       5   
4423              1                                    0                                    6                                       8   

      Curricular units 1st sem (approved)  Curricular units 1st sem (grade)  Curricular units 1st sem (without evaluations)  \
4419                                    5                         13.600000                                               0   
4420                                    6                         12.000000                                               0   
4421                                    7                         14.912500                                               0   
4422                                    5                         13.800000                                               0   
4423                                    6                         11.666667                                               0   

      Curricular units 2nd sem (credited)  Curricular units 2nd sem (enrolled)  Curricular units 2nd sem (evaluations)  \
4419                                    0                                    6                                       8   
4420                                    0                                    6                                       6   
4421                                    0                                    8                                       9   
4422                                    0                                    5                                       6   
4423                                    0                                    6                                       6   

      Curricular units 2nd sem (approved)  Curricular units 2nd sem (grade)  Curricular units 2nd sem (without evaluations)  Unemployment rate  \
4419                                    5                         12.666667                                               0               15.5   
4420                                    2                         11.000000                                               0               11.1   
4421                                    1                         13.500000                                               0               13.9   
4422                                    5                         12.000000                                               0                9.4   
4423                                    6                         13.000000                                               0               12.7   

      Inflation rate   GDP    Target  
4419             2.8 -4.06  Graduate  
4420             0.6  2.02   Dropout  
4421            -0.3  0.79   Dropout  
4422            -0.8 -3.12  Graduate  
4423             3.7 -1.70  Graduate

All of the columns appear to contain numbers except for the Target column which is a 3 class categorical text column that we will label encoded.
There are a number of label encoded columns that need to be one-hot encoded, such as Marital Status, Application mode, Course, *qualification, Mother/Father occupation, Nacionality (which should be corrected to Nationality).
There are columns that have the apostrophe punctuation mark. These might caused issues when referencing them in code so this will need to be removed.

In [6]:

# fix Nacionality column name typo to Nationality
data.rename(columns={'Nacionality': 'Nationality'}, inplace=True)
print("Sample\n", data['Nationality'].sample())

Sample
 1634    1
Name: Nationality, dtype: int64

In [7]:

# check for nulls in the columns
data.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 4424 entries, 0 to 4423
Data columns (total 37 columns):
 #   Column                                          Non-Null Count  Dtype  
---  ------                                          --------------  -----  
 0   Marital status                                  4424 non-null   int64  
 1   Application mode                                4424 non-null   int64  
 2   Application order                               4424 non-null   int64  
 3   Course                                          4424 non-null   int64  
 4   Daytime/evening attendance	                     4424 non-null   int64  
 5   Previous qualification                          4424 non-null   int64  
 6   Previous qualification (grade)                  4424 non-null   float64
 7   Nationality                                     4424 non-null   int64  
 8   Mother's qualification                          4424 non-null   int64  
 9   Father's qualification                          4424 non-null   int64  
 10  Mother's occupation                             4424 non-null   int64  
 11  Father's occupation                             4424 non-null   int64  
 12  Admission grade                                 4424 non-null   float64
 13  Displaced                                       4424 non-null   int64  
 14  Educational special needs                       4424 non-null   int64  
 15  Debtor                                          4424 non-null   int64  
 16  Tuition fees up to date                         4424 non-null   int64  
 17  Gender                                          4424 non-null   int64  
 18  Scholarship holder                              4424 non-null   int64  
 19  Age at enrollment                               4424 non-null   int64  
 20  International                                   4424 non-null   int64  
 21  Curricular units 1st sem (credited)             4424 non-null   int64  
 22  Curricular units 1st sem (enrolled)             4424 non-null   int64  
 23  Curricular units 1st sem (evaluations)          4424 non-null   int64  
 24  Curricular units 1st sem (approved)             4424 non-null   int64  
 25  Curricular units 1st sem (grade)                4424 non-null   float64
 26  Curricular units 1st sem (without evaluations)  4424 non-null   int64  
 27  Curricular units 2nd sem (credited)             4424 non-null   int64  
 28  Curricular units 2nd sem (enrolled)             4424 non-null   int64  
 29  Curricular units 2nd sem (evaluations)          4424 non-null   int64  
 30  Curricular units 2nd sem (approved)             4424 non-null   int64  
 31  Curricular units 2nd sem (grade)                4424 non-null   float64
 32  Curricular units 2nd sem (without evaluations)  4424 non-null   int64  
 33  Unemployment rate                               4424 non-null   float64
 34  Inflation rate                                  4424 non-null   float64
 35  GDP                                             4424 non-null   float64
 36  Target                                          4424 non-null   object 
dtypes: float64(7), int64(29), object(1)
memory usage: 1.2+ MB

The only text column is just the Target variable according to .info

All the columns have 4424 non-nulls which match the row count from .shape earlier.

Run the .isna also to make sure.

In [8]:

print("Total Null Values: ", data.isna().sum().sum())

Total Null Values:  0

No nulls, looks good

Now checking for duplicates

In [9]:

print(f"Duplicated rows: {data.duplicated().sum()}")

Duplicated rows: 0

No dupes

From earlier, removing apostrophes from column names e.g Mother's qualification

In [10]:

for column_name in data.columns:
    # Check if the column name contains an apostrophe
    if "'" in column_name:
        # Remove the apostrophe and update the column name
        new_column_name = column_name.replace("'", "")
        data.rename(columns={column_name: new_column_name}, inplace=True)

Check one row of data to confirm the removal of the apostrophe.

In [11]:

# check if the apostrophes were removed
print(data.sample())

    Marital status  Application mode  Application order  Course  Daytime/evening attendance\t  Previous qualification  \
38               4                39                  1    9991                             0                       1   

    Previous qualification (grade)  Nationality  Mothers qualification  Fathers qualification  Mothers occupation  Fathers occupation  \
38                           120.0            1                     19                     19                   9                   3   

    Admission grade  Displaced  Educational special needs  Debtor  Tuition fees up to date  Gender  Scholarship holder  Age at enrollment  \
38            113.9          1                          0       0                        1       0                   0                 39   

    International  Curricular units 1st sem (credited)  Curricular units 1st sem (enrolled)  Curricular units 1st sem (evaluations)  \
38              0                                    0                                    5                                       8   

    Curricular units 1st sem (approved)  Curricular units 1st sem (grade)  Curricular units 1st sem (without evaluations)  \
38                                    5                              11.0                                               0   

    Curricular units 2nd sem (credited)  Curricular units 2nd sem (enrolled)  Curricular units 2nd sem (evaluations)  \
38                                    0                                    5                                       5   

    Curricular units 2nd sem (approved)  Curricular units 2nd sem (grade)  Curricular units 2nd sem (without evaluations)  Unemployment rate  \
38                                    5                              11.6                                               0                7.6   

    Inflation rate   GDP    Target  
38             2.6  0.32  Graduate

Take a look at the number of unique values of each feature and sort in ascending order.

In [12]:

# Number of unique values per column in ascending order
data.nunique().sort_values()

Out [12]:

Daytime/evening attendance\t                        2
Displaced                                           2
Debtor                                              2
Educational special needs                           2
International                                       2
Scholarship holder                                  2
Gender                                              2
Tuition fees up to date                             2
Target                                              3
Marital status                                      6
Application order                                   8
Inflation rate                                      9
Curricular units 2nd sem (without evaluations)     10
Unemployment rate                                  10
GDP                                                10
Curricular units 1st sem (without evaluations)     11
Course                                             17
Previous qualification                             17
Application mode                                   18
Curricular units 2nd sem (credited)                19
Curricular units 2nd sem (approved)                20
Nationality                                        21
Curricular units 1st sem (credited)                21
Curricular units 2nd sem (enrolled)                22
Curricular units 1st sem (enrolled)                23
Curricular units 1st sem (approved)                23
Mothers qualification                              29
Curricular units 2nd sem (evaluations)             30
Mothers occupation                                 32
Fathers qualification                              34
Curricular units 1st sem (evaluations)             35
Fathers occupation                                 46
Age at enrollment                                  46
Previous qualification (grade)                    101
Admission grade                                   620
Curricular units 2nd sem (grade)                  782
Curricular units 1st sem (grade)                  797
dtype: int64

get a look at some of the data from the Curricular* columns to see if they are purely numeric or label encoded

In [13]:

data.filter(like='Curricular').sample(10)

Out [13]:

	Curricular units 1st sem (enrolled)	Curricular units 1st sem (evaluations)	Curricular units 1st sem (approved)	Curricular units 1st sem (grade)	Curricular units 1st sem (without evaluations)	Curricular units 2nd sem (enrolled)	Curricular units 2nd sem (evaluations)	Curricular units 2nd sem (approved)	Curricular units 2nd sem (grade)	Curricular units 2nd sem (without evaluations)
4081	5	9	2	11.500000	0	5	13	4	11.00	1
1630	5	9	4	12.250000	2	5	8	4	11.75	0
957	6	6	6	12.500000	0	6	7	5	13.00	0
2876	7	7	0	0.000000	0	5	10	0	0.00	0
1089	0	0	0	0.000000	0	0	0	0	0.00	0
1596	7	9	6	12.833333	0	8	12	6	12.50	0
1549	8	8	0	0.000000	1	8	8	0	0.00	1
3560	7	7	5	13.060000	0	7	9	0	0.00	0
3824	5	5	0	0.000000	0	5	6	1	15.00	0
2156	0	0	0	0.000000	0	0	0	0	0.00	0

These Curricular* columns don't appear to be categorical.

Look at some of the stats using the .describe function.

In [14]:

data.describe().T

Out [14]:

	count	mean	std	min	25%	50%	75%	max
Marital status	4424.0	1.178571	0.605747	1.00	1.00	1.000000	1.000000	6.000000
Application mode	4424.0	18.669078	17.484682	1.00	1.00	17.000000	39.000000	57.000000
Application order	4424.0	1.727848	1.313793	0.00	1.00	1.000000	2.000000	9.000000
Course	4424.0	8856.642631	2063.566416	33.00	9085.00	9238.000000	9556.000000	9991.000000
Daytime/evening attendance\t	4424.0	0.890823	0.311897	0.00	1.00	1.000000	1.000000	1.000000
Previous qualification	4424.0	4.577758	10.216592	1.00	1.00	1.000000	1.000000	43.000000
Previous qualification (grade)	4424.0	132.613314	13.188332	95.00	125.00	133.100000	140.000000	190.000000
Nationality	4424.0	1.873192	6.914514	1.00	1.00	1.000000	1.000000	109.000000
Mothers qualification	4424.0	19.561935	15.603186	1.00	2.00	19.000000	37.000000	44.000000
Fathers qualification	4424.0	22.275316	15.343108	1.00	3.00	19.000000	37.000000	44.000000
Mothers occupation	4424.0	10.960895	26.418253	0.00	4.00	5.000000	9.000000	194.000000
Fathers occupation	4424.0	11.032324	25.263040	0.00	4.00	7.000000	9.000000	195.000000
Admission grade	4424.0	126.978119	14.482001	95.00	117.90	126.100000	134.800000	190.000000
Displaced	4424.0	0.548373	0.497711	0.00	0.00	1.000000	1.000000	1.000000
Educational special needs	4424.0	0.011528	0.106760	0.00	0.00	0.000000	0.000000	1.000000
Debtor	4424.0	0.113698	0.317480	0.00	0.00	0.000000	0.000000	1.000000
Tuition fees up to date	4424.0	0.880651	0.324235	0.00	1.00	1.000000	1.000000	1.000000
Gender	4424.0	0.351718	0.477560	0.00	0.00	0.000000	1.000000	1.000000
Scholarship holder	4424.0	0.248418	0.432144	0.00	0.00	0.000000	0.000000	1.000000
Age at enrollment	4424.0	23.265145	7.587816	17.00	19.00	20.000000	25.000000	70.000000
International	4424.0	0.024864	0.155729	0.00	0.00	0.000000	0.000000	1.000000
Curricular units 1st sem (credited)	4424.0	0.709991	2.360507	0.00	0.00	0.000000	0.000000	20.000000
Curricular units 1st sem (enrolled)	4424.0	6.270570	2.480178	0.00	5.00	6.000000	7.000000	26.000000
Curricular units 1st sem (evaluations)	4424.0	8.299051	4.179106	0.00	6.00	8.000000	10.000000	45.000000
Curricular units 1st sem (approved)	4424.0	4.706600	3.094238	0.00	3.00	5.000000	6.000000	26.000000
Curricular units 1st sem (grade)	4424.0	10.640822	4.843663	0.00	11.00	12.285714	13.400000	18.875000
Curricular units 1st sem (without evaluations)	4424.0	0.137658	0.690880	0.00	0.00	0.000000	0.000000	12.000000
Curricular units 2nd sem (credited)	4424.0	0.541817	1.918546	0.00	0.00	0.000000	0.000000	19.000000
Curricular units 2nd sem (enrolled)	4424.0	6.232143	2.195951	0.00	5.00	6.000000	7.000000	23.000000
Curricular units 2nd sem (evaluations)	4424.0	8.063291	3.947951	0.00	6.00	8.000000	10.000000	33.000000
Curricular units 2nd sem (approved)	4424.0	4.435805	3.014764	0.00	2.00	5.000000	6.000000	20.000000
Curricular units 2nd sem (grade)	4424.0	10.230206	5.210808	0.00	10.75	12.200000	13.333333	18.571429
Curricular units 2nd sem (without evaluations)	4424.0	0.150316	0.753774	0.00	0.00	0.000000	0.000000	12.000000
Unemployment rate	4424.0	11.566139	2.663850	7.60	9.40	11.100000	13.900000	16.200000
Inflation rate	4424.0	1.228029	1.382711	-0.80	0.30	1.400000	2.600000	3.700000
GDP	4424.0	0.001969	2.269935	-4.06	-1.70	0.320000	1.790000	3.510000

From the .describe function there might be outliers with the following columns accorrding to how much jump the max values are compared to the 75 percentile:

Marital status - this is a nominal categorical variable so the percentile is not an outlier.
Application order - this is an ordinal categorical variable which has meaning in the percentile outlier
Age at enrollment - this makes sense since mostly the younger age group are the majority of students with only a small number in the higher age group
Curricular units 1st sem (credited)
Curricular units 1st sem (approved)
Curricular units 1st sem (without evaluations)
Curricular units 2nd sem (credited)
Curricular units 2nd sem (without evaluations)

Distribution Plot

Let's plot the numeric features onto histograms to further understand the distributions.

In [15]:

# plot out the numeric columns to view the distribution
numericColumns = data.loc[:, data.select_dtypes(include=['number']).columns]
numericColumns.shape

Out [15]:

(4424, 36)

There are 36 numercial columns to plot. Let's size the grid accordingly (12 x 3) and plot each column showing the mean and median line.

In [16]:

# plot layout of numeric columns
plt.subplots(12, 3, figsize=(20, 40)) # using a 4 x 7 grid of graphs since we are working with 28 columns; each graph will be 20x20 in size
plt.suptitle("Numeric Column Distributions", size=20)

for i , column in enumerate(numericColumns.columns, 1):
  plt.subplot(12, 3 , i)
  sns.histplot(numericColumns[column])

  plt.xticks(rotation=90)

  # titles and labels
  plt.title(column)
  plt.xlabel(column)
  plt.ylabel("Frequency")

  # include median and mean lines
  plt.axvline(data[column].mean(), c='blue', label='Mean')
  plt.axvline(data[column].median(), c='green', label='Median')
  plt.legend()

plt.tight_layout()
plt.show()

This helps in giving a picture of the distribution of all the numeric columns:

Marital Status is skewed towards Single.
Nationality is skewed towards Portuguese.

Somewhat normally distributed features are:

Previous qualification (grade)
Admission grade
Circular units 1st & 2nd sem (grade) (but with predominance on zero's)

Binary features:

Daytime/evening attendance (majority in Daytime)
Displaced (close to balanced but Displaced is dominant)
Educational special needs (mostly no) Debtor (mostly no)
Tuition fees up to date (majority are)
Gender (female are close to twice as much as male)
Scholarship holder (majority no)
International (very small fraction are)

Unemployment rate, Inflation rate, GDP are probably specific to the student's country. According to the .nunique results earlier there seems to be 10 different countries

Age of enrollment seems to have a natural progression where the majority age of students are 18-20 and it tapers down as the age increases

There are a few numerical variables that are categorical and have high cardinality which needs to be addressed to minimize the features.

Application mode
Application order
Course
Fathers occupation
Mothers occupation
Nationality
Marital status
Previous qualification
Fathers qualification
Mothers qualification

I will attempt to use binning or some other encoding where possible on these later.

Going back to see if Unemployment rate, Infation rate, and GDP are all specific to the country of where the student resides, let's look at the combination of each.

In [17]:

unique_Country_Combinations = data[['Unemployment rate', 'Inflation rate', 'GDP']].drop_duplicates().sort_index()
print('Total Unique Rows:', unique_Country_Combinations.shape[0])
print(unique_Country_Combinations)

Total Unique Rows: 10
    Unemployment rate  Inflation rate   GDP
0                10.8             1.4  1.74
1                13.9            -0.3  0.79
3                 9.4            -0.8 -3.12
5                16.2             0.3 -0.92
6                15.5             2.8 -4.06
9                 8.9             1.4  3.51
11               12.7             3.7 -1.70
20               11.1             0.6  2.02
25                7.6             2.6  0.32
33               12.4             0.5  1.79

So the 3 columns Unemployment rate, Infation rate, and GDP are all related to the same country. With a total of 10 of them.

Target Variable

What's the distribution of our target variable like. Since the target variable is the only string (or object type), we'll use .describe and only include object type.

In [18]:

data.describe(include='object')

Out [18]:

	Target
count	4424
unique	3
top	Graduate
freq	2209

Our target has no nulls and has 3 unique values. The most frequent is Graduate and that has 2209 instances in the data. Now let's plot the distribution of the 3 values.

In [19]:

sns.countplot(x='Target', data=data)
plt.title('Distribution of Target Classes')

# show percentage of each Target value on plot
total = len(data)
for p in plt.gca().patches:
    height = p.get_height()
    plt.text(p.get_x() + p.get_width() / 2.,
             height + 0.1,
             '{:.1f}%'.format((height / total) * 100),
             ha="center")

plt.show()

As described on the website source of this data, there are 3 classes on the categorical Target variable. The Graduate value is the most at close to 50% of the data. Dropout at 32% is 2nd and then Enrolled is place with 17%.

Essentially for modelling this needs to be a numerical column, label encoded. Let's do that now and use the ID maps below:

Dropout - 1 - 32.1%
Graduate - 2 - 49.9%
Enrolled - 3 - 17.9%

We'll update Dropout to 1, Graduate to 2 and Enrolled to 3.

In [20]:

from sklearn.preprocessing import LabelEncoder

class_mapping = {'Graduate': 2, 'Enrolled': 3, 'Dropout': 1}

data['TargetEncoded'] = data['Target'].map(class_mapping)
print('Sample rows of Target variable: ', data['TargetEncoded'].sample(10))

Sample rows of Target variable:  3377    2
1297    1
3467    2
1444    2
4036    2
826     1
2124    2
1891    2
4012    1
1928    3
Name: TargetEncoded, dtype: int64

The sample of the Target variable above confirms that it has been remapped to numeric values.

Plot the new target column just to check.

In [21]:

sns.countplot(x='TargetEncoded', data=data)
plt.title('Distribution of Target Classes')

# show percentage of each Target value on plot
total = len(data)
for p in plt.gca().patches:
    height = p.get_height()
    plt.text(p.get_x() + p.get_width() / 2.,
             height + 0.1,
             '{:.1f}%'.format((height / total) * 100),
             ha="center")

plt.show()

Looks good and identical to the original Target columns. So we will now drop the original column.

In [22]:

data.drop(columns=['Target'], inplace=True)

In [23]:

# Check that the original Target columns is removed
print(data.sample())

      Marital status  Application mode  Application order  Course  Daytime/evening attendance\t  Previous qualification  \
4134               1                 1                  1    9238                             1                       1   

      Previous qualification (grade)  Nationality  Mothers qualification  Fathers qualification  Mothers occupation  Fathers occupation  \
4134                           142.0            1                     34                     38                   4                   7   

      Admission grade  Displaced  Educational special needs  Debtor  Tuition fees up to date  Gender  Scholarship holder  Age at enrollment  \
4134            127.3          0                          0       0                        1       0                   1                 18   

      International  Curricular units 1st sem (credited)  Curricular units 1st sem (enrolled)  Curricular units 1st sem (evaluations)  \
4134              0                                    0                                    6                                      12   

      Curricular units 1st sem (approved)  Curricular units 1st sem (grade)  Curricular units 1st sem (without evaluations)  \
4134                                    4                              11.0                                               0   

      Curricular units 2nd sem (credited)  Curricular units 2nd sem (enrolled)  Curricular units 2nd sem (evaluations)  \
4134                                    0                                    6                                       9   

      Curricular units 2nd sem (approved)  Curricular units 2nd sem (grade)  Curricular units 2nd sem (without evaluations)  Unemployment rate  \
4134                                    6                              11.5                                               0                9.4   

      Inflation rate   GDP  TargetEncoded  
4134            -0.8 -3.12              2

I will also take the next step and convert this 3 class Target variable into binary since Graduate and Enrolled can be seen as the same in terms of a student dropping out or not. Both are considered Not Dropped. So we will combine Graduate (2) and Enrolled (3) into one bucket. To make it simple I will update both Graduate (2) and Enrolled (3) to 0. This will convert the TargetEncoded variable to binary where having the 0 value represents Not Dropped while 1 means Dropped.

In [24]:

# Update values 2, 3 to 0
data.loc[data['TargetEncoded'].isin([2, 3]), 'TargetEncoded'] = 0
# check and print count of all unique values
print('\nCount Breakdown of Target:\n', data['TargetEncoded'].value_counts())
print('\nPercentage Breakdown of Target:\n', data['TargetEncoded'].value_counts(normalize=True) * 100)
data.head(10)

Out [24]:

Count Breakdown of Target:
 TargetEncoded
0    3003
1    1421
Name: count, dtype: int64

Percentage Breakdown of Target:
 TargetEncoded
0    67.879747
1    32.120253
Name: proportion, dtype: float64

	Marital status	Application mode	Application order	Course	Daytime/evening attendance\t	Previous qualification	Previous qualification (grade)	Nationality	Mothers qualification	Fathers qualification	Mothers occupation	Fathers occupation	Admission grade	Displaced	Debtor	Tuition fees up to date	Gender	Scholarship holder	Age at enrollment	International	Curricular units 1st sem (enrolled)	Curricular units 1st sem (evaluations)	Curricular units 1st sem (approved)	Curricular units 1st sem (grade)	Curricular units 2nd sem (enrolled)	Curricular units 2nd sem (evaluations)	Curricular units 2nd sem (approved)	Curricular units 2nd sem (grade)	Curricular units 2nd sem (without evaluations)	Unemployment rate	Inflation rate	GDP	TargetEncoded
0	1	17	5	171	1	1	122.0	1	19	12	5	9	127.3	1	0	1	1	0	20	0	0	0	0	0.000000	0	0	0	0.000000	0	10.8	1.4	1.74	1
1	1	15	1	9254	1	1	160.0	1	1	3	3	3	142.5	1	0	0	1	0	19	0	6	6	6	14.000000	6	6	6	13.666667	0	13.9	-0.3	0.79	0
2	1	1	5	9070	1	1	122.0	1	37	37	9	9	124.8	1	0	0	1	0	19	0	6	0	0	0.000000	6	0	0	0.000000	0	10.8	1.4	1.74	1
3	1	17	2	9773	1	1	122.0	1	38	37	5	3	119.6	1	0	1	0	0	20	0	6	8	6	13.428571	6	10	5	12.400000	0	9.4	-0.8	-3.12	0
4	2	39	1	8014	0	1	100.0	1	37	38	9	9	141.5	0	0	1	0	0	45	0	6	9	5	12.333333	6	6	6	13.000000	0	13.9	-0.3	0.79	0
5	2	39	1	9991	0	19	133.1	1	37	37	9	7	114.8	0	1	1	1	0	50	0	5	10	5	11.857143	5	17	5	11.500000	5	16.2	0.3	-0.92	0
6	1	1	1	9500	1	1	142.0	1	19	38	7	10	128.4	1	0	1	0	1	18	0	7	9	7	13.300000	8	8	8	14.345000	0	15.5	2.8	-4.06	0
7	1	18	4	9254	1	1	119.0	1	37	37	9	9	113.1	1	0	0	1	0	22	0	5	5	0	0.000000	5	5	0	0.000000	0	15.5	2.8	-4.06	1
8	1	1	3	9238	1	1	137.0	62	1	1	9	9	129.3	0	0	1	0	1	21	1	6	8	6	13.875000	6	7	6	14.142857	0	16.2	0.3	-0.92	0
9	1	1	1	9238	1	1	138.0	1	1	19	4	7	123.0	1	1	0	0	0	18	0	6	9	5	11.400000	6	14	2	13.500000	0	8.9	1.4	3.51	1

The above data now shows 2 values (0 and 1) for the Target variable making this a binary classification.

Let's plot a graph for better visualtization.

In [25]:

from matplotlib.lines import Line2D

sns.countplot(x='TargetEncoded', data=data)
plt.title('Distribution of Updated Target Classes')

# show percentage of each Target value on plot
total = len(data)
for p in plt.gca().patches:
    height = p.get_height()
    plt.text(p.get_x() + p.get_width() / 2.,
             height + 0.1,
             '{:.1f}%'.format((height / total) * 100),
             ha="center")
    # Add custom legend labels

plt.show()

The target variable has been binarized.

Target variable Count and Percentage breakdown:

TargetEncoded	Count	Percentage
0 - Not Dropped	3003	67.88%
1 - Dropped	1421	32.12%

The ratio between Not Dropped and Dropped is about 70 : 30 indicating an imbalance. Imbalanced datasets can pose challenges for machine learning modeling, since some algorithms assume balanced class distribution or have difficulty learning from minority class samples. This can be remedied by certain techniques such as resampling.

Target-Variable-Count-Plot-Across-Features

Now let's look at the Count plot of the TargetEncoded variable across all the other features. The graph on the right are the drop out targets. It will not be sharing the y axis values to better see how the variable differs with each target value whether it's the dropped out population or not. Granted some of the variables will not make sense with this plot such as those showing grades or credits.

In [26]:

i = 1
for col in data.drop(columns=['TargetEncoded']):
        g = sns.catplot(x = col, kind='count', col = 'TargetEncoded', data=data, sharey=True)
        g.figure.suptitle(i)
        g.set_xticklabels()
        i += 1

From plotting the Target against each feature we can observe the following:

In Application mode there's a stark rise under the Dropped population for value: 39 - Over 23 years old.
It's hard to see the exact value due to the high unique values but for Course, the far right bar plot in pink (9991 - Management evening attendance) shows a much higher count for Dropped.
The Displaced comparison with the Target seems surprising. There's a lot more Displaced that have not dropped out than dropped.
Debtor shows a higher count for those that are dropped.
Same for those with Tuition fees that are NOT up to date. It's higher for the dropped.
For Gender, the amount of dropped between male and female pretty balanced. There are much more female enrolled though compared to males, more than twice the amount.

High Percentage of Dropped across Variable Values

Another look to possibly see some patterns with the Dropped Target is to calculate what percentage of a category value are Dropped vs Not Dropped. Plot only the ones that have a higher percentage when Dropped. Let's give a threshold of when Dropped percentage is above 65. Let's loop through the columns and only show the values of the features that have a high Dropped percentage compared to Not Dropped.

In [27]:

i = 1

for col in data.drop(columns=['TargetEncoded']):

        # Calculating the percentage of dropouts by col
        result = data.groupby(col)['TargetEncoded'].value_counts(normalize=True).mul(100).unstack()
        result = result.fillna(0)
        plotit = result[result[1] > 65]
        # print(plotit.sort_values(by=1, ascending=False))

        if plotit.empty == False:
                # Plotting
                plotit.plot(kind='bar', figsize=(5, 3))
                plt.title('Percentage of Target 0 and Target 1 by ' + column_name)
                plt.xlabel(col)
                plt.ylabel('Percentage')
                plt.xticks(rotation=0)
                plt.grid(True)
                plt.legend(title='Target')

                # Show the plot
                plt.show()

Not all columns plotted are applicable such as Curricular units or any other continuous numeric values.

But some things to note from the above plots:

for Marital status: 6 - legally separated has a high percentage of dropped compared to not dropped.
in Application mode below have high percentages for Dropped also - 2 - Ordinance No. 612/93, - 26 - Ordinance No. 533-A/99, item b2 (Different Plan) - 27 - Ordinance No. 533-A/99, item b3 (Other Institution)
for Course: those taking 33 - Biofuel Production Technologies have a higher likelihood of dropping out.
Age of Enrollment: ages 29, 31, 38, 51, 58, 59, 61, 70 mostly the higher the age the chance of dropping out increases
For both parents' occupation: 90 - Other Situation, 99 - (blank) are high risk.

Some of these might be anomalies but it's worth pointing out.

Target Variable Imbalance

Let's address the imbalance of the target variable that we pointed out earlier. Since the target of Not Dropped Out (0) vs Dropped (1) has a ratio of 70% to 30%, we will use upsampling of the minority class.

In [28]:

from sklearn.utils import resample

# Separate majority and minority classes
majority_class = data[data['TargetEncoded'] == 0]
minority_class = data[data['TargetEncoded'] == 1]

In [29]:

display(majority_class.head())
print('Shape of Majority Class:', majority_class.shape)
print('Shape of Minority Class:', minority_class.shape)

	Marital status	Application mode	Application order	Course	Daytime/evening attendance\t	Previous qualification	Previous qualification (grade)	Nationality	Mothers qualification	Fathers qualification	Mothers occupation	Fathers occupation	Admission grade	Displaced	Debtor	Tuition fees up to date	Gender	Scholarship holder	Age at enrollment	Curricular units 1st sem (enrolled)	Curricular units 1st sem (evaluations)	Curricular units 1st sem (approved)	Curricular units 1st sem (grade)	Curricular units 2nd sem (enrolled)	Curricular units 2nd sem (evaluations)	Curricular units 2nd sem (approved)	Curricular units 2nd sem (grade)	Curricular units 2nd sem (without evaluations)	Unemployment rate	Inflation rate	GDP
1	1	15	1	9254	1	1	160.0	1	1	3	3	3	142.5	1	0	0	1	0	19	6	6	6	14.000000	6	6	6	13.666667	0	13.9	-0.3	0.79
3	1	17	2	9773	1	1	122.0	1	38	37	5	3	119.6	1	0	1	0	0	20	6	8	6	13.428571	6	10	5	12.400000	0	9.4	-0.8	-3.12
4	2	39	1	8014	0	1	100.0	1	37	38	9	9	141.5	0	0	1	0	0	45	6	9	5	12.333333	6	6	6	13.000000	0	13.9	-0.3	0.79
5	2	39	1	9991	0	19	133.1	1	37	37	9	7	114.8	0	1	1	1	0	50	5	10	5	11.857143	5	17	5	11.500000	5	16.2	0.3	-0.92
6	1	1	1	9500	1	1	142.0	1	19	38	7	10	128.4	1	0	1	0	1	18	7	9	7	13.300000	8	8	8	14.345000	0	15.5	2.8	-4.06

Shape of Majority Class: (3003, 37)
Shape of Minority Class: (1421, 37)

Separate the Dropped (1) vs Not Dropped (0) its own dataframe. Then apply the resample function to create an upsampled minority. Then add the upsample minority with the majority class.

In [30]:

# Upsample minority class
minority_upsampled = resample(minority_class,
                              replace=True,  # Sample with replacement
                              n_samples=len(majority_class),  # Match number of majority class
                              random_state=42)  # Reproducible results
display(minority_upsampled.head())
print('Minority Upsampled:', minority_upsampled.shape)

	Marital status	Application mode	Application order	Course	Daytime/evening attendance\t	Previous qualification	Previous qualification (grade)	Nationality	Mothers qualification	Fathers qualification	Mothers occupation	Fathers occupation	Admission grade	Displaced	Debtor	Tuition fees up to date	Gender	Scholarship holder	Age at enrollment	Curricular units 1st sem (credited)	Curricular units 1st sem (enrolled)	Curricular units 1st sem (evaluations)	Curricular units 1st sem (approved)	Curricular units 1st sem (grade)	Curricular units 1st sem (without evaluations)	Curricular units 2nd sem (credited)	Curricular units 2nd sem (enrolled)	Curricular units 2nd sem (evaluations)	Curricular units 2nd sem (approved)	Curricular units 2nd sem (grade)	Curricular units 2nd sem (without evaluations)	Unemployment rate	Inflation rate	GDP	TargetEncoded
3564	1	44	1	9119	1	39	130.0	1	19	3	3	4	130.0	0	1	0	1	0	23	0	5	5	0	0.000000	0	0	5	5	0	0.00000	0	15.5	2.8	-4.06	1
2760	1	39	1	9500	1	4	150.0	1	3	38	3	7	127.0	1	0	1	1	0	26	2	7	8	6	13.046667	0	2	8	10	7	13.30625	0	11.1	0.6	2.02	1
4015	4	39	1	9991	0	1	133.1	1	37	37	5	5	138.7	0	0	1	0	0	31	0	5	7	3	10.333333	0	0	5	6	3	12.00000	0	10.8	1.4	1.74	1
3577	1	42	1	9500	1	1	100.0	1	19	19	9	9	100.0	0	1	1	1	0	24	0	8	16	2	10.250000	1	0	8	16	2	10.25000	1	12.7	3.7	-1.70	1
3501	1	39	1	9070	1	1	140.0	1	37	37	9	10	118.0	0	1	1	1	1	45	0	6	6	6	13.500000	0	0	6	6	0	0.00000	0	16.2	0.3	-0.92	1

Minority Upsampled: (3003, 37)

First confirm the TargetEncoded distribution count.

In [31]:

data['TargetEncoded'].value_counts()

Out [31]:

TargetEncoded
0    3003
1    1421
Name: count, dtype: int64

Now combine the upsampled minority with the majority population. This will be the new dataset we will work on.

In [32]:

# Combine majority class with upsampled minority class
balanced_data = pd.concat([majority_class, minority_upsampled])

# Now 'balanced_data' contains an upsampled distribution of the minority class

Compare the TargetEncoded value counts with the new unbalanced_data.

In [33]:

display(balanced_data.head())
print('Shape of unbalanced_data:', balanced_data.shape)
print('Distribution of unbalanced_data:', balanced_data['TargetEncoded'].value_counts())

	Marital status	Application mode	Application order	Course	Daytime/evening attendance\t	Previous qualification	Previous qualification (grade)	Nationality	Mothers qualification	Fathers qualification	Mothers occupation	Fathers occupation	Admission grade	Displaced	Debtor	Tuition fees up to date	Gender	Scholarship holder	Age at enrollment	Curricular units 1st sem (enrolled)	Curricular units 1st sem (evaluations)	Curricular units 1st sem (approved)	Curricular units 1st sem (grade)	Curricular units 2nd sem (enrolled)	Curricular units 2nd sem (evaluations)	Curricular units 2nd sem (approved)	Curricular units 2nd sem (grade)	Curricular units 2nd sem (without evaluations)	Unemployment rate	Inflation rate	GDP
1	1	15	1	9254	1	1	160.0	1	1	3	3	3	142.5	1	0	0	1	0	19	6	6	6	14.000000	6	6	6	13.666667	0	13.9	-0.3	0.79
3	1	17	2	9773	1	1	122.0	1	38	37	5	3	119.6	1	0	1	0	0	20	6	8	6	13.428571	6	10	5	12.400000	0	9.4	-0.8	-3.12
4	2	39	1	8014	0	1	100.0	1	37	38	9	9	141.5	0	0	1	0	0	45	6	9	5	12.333333	6	6	6	13.000000	0	13.9	-0.3	0.79
5	2	39	1	9991	0	19	133.1	1	37	37	9	7	114.8	0	1	1	1	0	50	5	10	5	11.857143	5	17	5	11.500000	5	16.2	0.3	-0.92
6	1	1	1	9500	1	1	142.0	1	19	38	7	10	128.4	1	0	1	0	1	18	7	9	7	13.300000	8	8	8	14.345000	0	15.5	2.8	-4.06

Shape of unbalanced_data: (6006, 37)
Distribution of unbalanced_data: TargetEncoded
0    3003
1    3003
Name: count, dtype: int64

So the new variable with the upsampled data confirms that the distribution of each TargetEncoded value are equal.

Let's plot to confirm visually.

In [34]:

sns.countplot(x='TargetEncoded', data=balanced_data)
plt.title('Distribution of Updated Target Classes')

# show percentage of each Target value on plot
total = len(data)
for p in plt.gca().patches:
    height = p.get_height()
    plt.text(p.get_x() + p.get_width() / 2.,
             height + 0.1,
             '{:.1f}%'.format((height / total) * 100),
             ha="center")
    # Add custom legend labels

plt.show()

Looks good. Now I'd like to keep name of the main dataframe that we'll be working on to be called data too so I'll set the unbalanced_data to be the new data variable.

In [35]:

data = balanced_data

Confirm by plotting the same TargetEncoded distribution but using data variable.

In [36]:

sns.countplot(x='TargetEncoded', data=data)
plt.title('Distribution of Updated Target Classes')

# show percentage of each Target value on plot
total = len(data)
for p in plt.gca().patches:
    height = p.get_height()
    plt.text(p.get_x() + p.get_width() / 2.,
             height + 0.1,
             '{:.1f}%'.format((height / total) * 100),
             ha="center")
    # Add custom legend labels

plt.show()

Binning Categorical Variables

Using the list of these categorical columns we identified above earlier, we'll go through each one and address appropriately.

Application mode
Application order
Course
Fathers occupation
Mothers occupation
Nationality
Marital status
Previous qualification
Fathers qualification
Mothers qualification

Let's start with the Qualification columns.

Qualification Variables

For example, in the Qualification columns used by Previous qualification, Mothers qualification and Fathers qualification, there are 34 different possible values that can be grouped to similar qualifications. Perhaps group them into 9 levels starting from primary education level all the way up to higher education. From research and to the best of my knowledge, I bunched each of the the 34 values to the most comparable ones.

There would need to be buckets for Vocational and the incomplete ones, Unknown and Nonliterate.

The mapping chart below is the result of the grouping. Using this as a guideline, we'll use the functions .map to accomplish this.

Original Qualification: 34 Distinct Values	New Bin Value	New (Binned) Qualification Values: 9 Distinct Values and its ID Value
1 - Secondary Education - 12th Year of Schooling or Equivalent	Secondary - 7	Unknown - 1
2 - Higher Education - Bachelor's Degree	Higher - 9	Nonliterate - 2
3 - Higher Education - Degree	Higher - 9	Elementary - 3
4 - Higher Education - Master's	Higher - 9	Middle School Incomplete - 4
5 - Higher Education - Doctorate	Higher - 9	Middle School - 5
6 - Frequency of Higher Education	Higher - 9	Secondary Incomplete - 6
9 - 12th Year of Schooling - Not Completed	Secondary Incomplete - 6	Secondary - 7
10 - 11th Year of Schooling - Not Completed	Secondary Incomplete - 6	Vocational - 8
11 - 7th Year (Old)	Middle School - 5	Higher - 9
12 - Other - 11th Year of Schooling	Secondary - 7
13 - 2nd year complementary high school course	Secondary - 7
14 - 10th Year of Schooling	Secondary - 7
15 - 10th year of schooling - not completed	Secondary Incomplete - 6
18 - General commerce course	Higher - 9
19 - Basic Education 3rd Cycle (9th/10th/11th Year) or Equivalent	Secondary Incomplete - 6
20 - Complementary High School Course	Secondary - 7
22 - Technical-professional course	Vocational - 8
25 - Complementary High School Course - not concluded	Secondary Incomplete - 6
26 - 7th year of schooling	Middle School - 5
27 - 2nd cycle of the general high school course	Secondary Incomplete - 6
29 - 9th Year of Schooling - Not Completed	Secondary Incomplete - 6
30 - 8th year of schooling	Middle School Incomplete - 4
31 - General Course of Administration and Commerce	Higher - 9
33 - Supplementary Accounting and Administration	Vocational - 8
34 - Unknown	Unknown - 1
35 - Can't read or write	Nonliterate - 2
36 - Can read without having a 4th year of schooling	Nonliterate - 2
37 - Basic education 1st cycle (4th/5th year) or equivalent	Elementary - 3
38 - Basic Education 2nd Cycle (6th/7th/8th Year) or Equivalent	Middle School - 5
39 - Technological specialization course	Vocational - 8
40 - Higher education - degree (1st cycle)	Secondary - 7
41 - Specialized higher studies course	Higher - 9
42 - Professional higher technical course	Higher - 9
43 - Higher Education - Master (2nd cycle)	Higher - 9
44 - Higher Education - Doctorate (3rd cycle)	Higher - 9

I'll store this maping chart above into a variable called qualification_bin_mapping.

In [37]:

qualification_bin_mapping = {
            1:7,
            2:9,
            3:9,
            4:9,
            5:9,
            6:9,
            9:6,
            10:6,
            11:5,
            12:7,
            13:7,
            14:7,
            15:6,
            18:9,
            19:6,
            20:7,
            22:8,
            25:6,
            26:5,
            27:6,
            29:6,
            30:4,
            31:9,
            33:8,
            34:1,
            35:2,
            36:2,
            37:3,
            38:5,
            39:8,
            40:7,
            41:9,
            42:9,
            43:9,
            44:9
}

display(qualification_bin_mapping)

{1: 7,
 2: 9,
 3: 9,
 4: 9,
 5: 9,
 6: 9,
 9: 6,
 10: 6,
 11: 5,
 12: 7,
 13: 7,
 14: 7,
 15: 6,
 18: 9,
 19: 6,
 20: 7,
 22: 8,
 25: 6,
 26: 5,
 27: 6,
 29: 6,
 30: 4,
 31: 9,
 33: 8,
 34: 1,
 35: 2,
 36: 2,
 37: 3,
 38: 5,
 39: 8,
 40: 7,
 41: 9,
 42: 9,
 43: 9,
 44: 9}

From the above output, we're assured the chart is implemented and can go ahead and map the 3 columns: Previous qualification, Mothers qualification and Fathers qualification. Apply the mapping to these columns in the DataFrame.

In [38]:

# Apply the mapping to the appropriate columns in the DataFrame
data['Previous qualification Bin'] = data['Previous qualification'].map(qualification_bin_mapping)
data['Mothers qualification Bin'] = data['Mothers qualification'].map(qualification_bin_mapping)
data['Fathers qualification Bin'] = data['Fathers qualification'].map(qualification_bin_mapping)

Confirm below to see the change in the dataframe using .info

In [39]:

data.info()

<class 'pandas.core.frame.DataFrame'>
Index: 6006 entries, 1 to 2393
Data columns (total 40 columns):
 #   Column                                          Non-Null Count  Dtype  
---  ------                                          --------------  -----  
 0   Marital status                                  6006 non-null   int64  
 1   Application mode                                6006 non-null   int64  
 2   Application order                               6006 non-null   int64  
 3   Course                                          6006 non-null   int64  
 4   Daytime/evening attendance	                     6006 non-null   int64  
 5   Previous qualification                          6006 non-null   int64  
 6   Previous qualification (grade)                  6006 non-null   float64
 7   Nationality                                     6006 non-null   int64  
 8   Mothers qualification                           6006 non-null   int64  
 9   Fathers qualification                           6006 non-null   int64  
 10  Mothers occupation                              6006 non-null   int64  
 11  Fathers occupation                              6006 non-null   int64  
 12  Admission grade                                 6006 non-null   float64
 13  Displaced                                       6006 non-null   int64  
 14  Educational special needs                       6006 non-null   int64  
 15  Debtor                                          6006 non-null   int64  
 16  Tuition fees up to date                         6006 non-null   int64  
 17  Gender                                          6006 non-null   int64  
 18  Scholarship holder                              6006 non-null   int64  
 19  Age at enrollment                               6006 non-null   int64  
 20  International                                   6006 non-null   int64  
 21  Curricular units 1st sem (credited)             6006 non-null   int64  
 22  Curricular units 1st sem (enrolled)             6006 non-null   int64  
 23  Curricular units 1st sem (evaluations)          6006 non-null   int64  
 24  Curricular units 1st sem (approved)             6006 non-null   int64  
 25  Curricular units 1st sem (grade)                6006 non-null   float64
 26  Curricular units 1st sem (without evaluations)  6006 non-null   int64  
 27  Curricular units 2nd sem (credited)             6006 non-null   int64  
 28  Curricular units 2nd sem (enrolled)             6006 non-null   int64  
 29  Curricular units 2nd sem (evaluations)          6006 non-null   int64  
 30  Curricular units 2nd sem (approved)             6006 non-null   int64  
 31  Curricular units 2nd sem (grade)                6006 non-null   float64
 32  Curricular units 2nd sem (without evaluations)  6006 non-null   int64  
 33  Unemployment rate                               6006 non-null   float64
 34  Inflation rate                                  6006 non-null   float64
 35  GDP                                             6006 non-null   float64
 36  TargetEncoded                                   6006 non-null   int64  
 37  Previous qualification Bin                      6006 non-null   int64  
 38  Mothers qualification Bin                       6006 non-null   int64  
 39  Fathers qualification Bin                       6006 non-null   int64  
dtypes: float64(7), int64(33)
memory usage: 1.9 MB

Now remove the 3 original qualtification columns: Previous qualification, Mothers qualification and Fathers qualification.

In [40]:

data.drop('Previous qualification', axis=1, inplace=True)
data.drop('Mothers qualification', axis=1, inplace=True)
data.drop('Fathers qualification', axis=1, inplace=True)

Checking to that the 3 columns are removed by looking at some data.

In [41]:

data.head(15)

Out [41]:

	Marital status	Application mode	Application order	Course	Daytime/evening attendance\t	Previous qualification (grade)	Nationality	Mothers occupation	Fathers occupation	Admission grade	Displaced	Debtor	Tuition fees up to date	Gender	Scholarship holder	Age at enrollment	International	Curricular units 1st sem (enrolled)	Curricular units 1st sem (evaluations)	Curricular units 1st sem (approved)	Curricular units 1st sem (grade)	Curricular units 1st sem (without evaluations)	Curricular units 2nd sem (enrolled)	Curricular units 2nd sem (evaluations)	Curricular units 2nd sem (approved)	Curricular units 2nd sem (grade)	Curricular units 2nd sem (without evaluations)	Unemployment rate	Inflation rate	GDP	Previous qualification Bin	Mothers qualification Bin	Fathers qualification Bin
1	1	15	1	9254	1	160.0	1	3	3	142.5	1	0	0	1	0	19	0	6	6	6	14.000000	0	6	6	6	13.666667	0	13.9	-0.3	0.79	7	7	9
3	1	17	2	9773	1	122.0	1	5	3	119.6	1	0	1	0	0	20	0	6	8	6	13.428571	0	6	10	5	12.400000	0	9.4	-0.8	-3.12	7	5	3
4	2	39	1	8014	0	100.0	1	9	9	141.5	0	0	1	0	0	45	0	6	9	5	12.333333	0	6	6	6	13.000000	0	13.9	-0.3	0.79	7	3	5
5	2	39	1	9991	0	133.1	1	9	7	114.8	0	1	1	1	0	50	0	5	10	5	11.857143	0	5	17	5	11.500000	5	16.2	0.3	-0.92	6	3	3
6	1	1	1	9500	1	142.0	1	7	10	128.4	1	0	1	0	1	18	0	7	9	7	13.300000	0	8	8	8	14.345000	0	15.5	2.8	-4.06	7	6	5
8	1	1	3	9238	1	137.0	62	9	9	129.3	0	0	1	0	1	21	1	6	8	6	13.875000	0	6	7	6	14.142857	0	16.2	0.3	-0.92	7	7	7
10	1	1	1	9670	1	139.0	1	5	7	130.6	1	0	1	0	0	18	0	6	6	6	12.333333	0	6	7	5	14.200000	0	13.9	-0.3	0.79	7	5	6
11	1	1	1	9500	1	136.0	1	9	9	119.3	1	0	1	0	1	18	0	8	8	7	13.214286	0	8	8	7	13.214286	0	12.7	3.7	-1.70	7	6	5
13	1	53	1	9254	1	110.0	1	4	7	111.8	1	0	1	0	1	21	0	6	7	6	10.571429	0	6	8	5	11.000000	0	8.9	1.4	3.51	9	7	7
14	1	1	1	9085	1	149.0	1	5	5	137.1	1	0	1	0	1	18	0	5	7	4	13.250000	0	5	5	5	12.000000	0	10.8	1.4	1.74	7	5	3
16	1	18	1	9238	1	137.0	1	5	8	137.4	1	0	1	0	0	18	0	6	10	1	12.000000	0	6	14	2	11.000000	0	10.8	1.4	1.74	7	6	5
17	1	17	2	9500	1	135.0	1	5	4	127.3	1	0	1	0	0	18	0	7	8	7	13.306250	0	8	8	8	14.545000	0	15.5	2.8	-4.06	7	6	7
18	1	1	1	9130	1	137.0	1	3	5	136.3	1	0	1	0	0	20	0	5	8	4	12.500000	1	5	8	4	12.250000	2	10.8	1.4	1.74	7	9	6
19	1	1	1	9853	1	140.0	1	7	7	124.6	1	0	1	0	0	18	0	7	7	6	11.666667	0	7	8	6	13.500000	0	16.2	0.3	-0.92	7	6	6
20	1	1	3	171	1	122.0	1	9	8	120.3	0	0	1	0	1	21	0	0	0	0	0.000000	0	0	0	0	0.000000	0	11.1	0.6	2.02	7	7	7

Marital Status Variable

Let's look at how the values are distributed in Marital Status.

In [42]:

data['Marital status'].value_counts(normalize=True, ascending=False).mul(100).round(2)

Out [42]:

Marital status
1    87.16
2     9.49
4     2.35
5     0.72
6     0.22
3     0.07
Name: proportion, dtype: float64

THe most frequent at 87.16% is 1 - Single, then down to 9.49% for 2 - Married, then 2.35% for 4 - Divorced, and then less than 1% for 5 - Facto Union, 6 - Legally Separated, and 3 - Widower.

Let's bin them to have Single, Married and Divorced on its own bin and the last 3 (Facto Union, Legally Separated, Widower) are bundled together into Other.

Since the current values for Marital Status is from 1-6, I will use 7 to be the Other. I will update the values 5, 6, and 3 to be 7. Refer to the chart below.

Marital Status	Percentage Frequency	New Bin Value	End Result
1 - single	87.16	1	Single
2 - married	9.49	2	Married
4 - divorced	2.35	4	Divorced
5 - facto union	0.72	7	Other
6 - legally separated	0.22	7	Other
3 - widower	0.07	7	Other

We'll create a mapping variable below to imitate the mapping chart above.

In [43]:

marital_bin_mapping = {
    1:1,
    2:2,
    3:7,
    4:4,
    5:7,
    6:7
}

display(marital_bin_mapping)

{1: 1, 2: 2, 3: 7, 4: 4, 5: 7, 6: 7}

Using the variable above we can go ahead and map the Marital status and create a new one called Marital Status Bin.

In [44]:

# Apply the mapping to the appropriate column
data['Marital Status Bin'] = data['Marital status'].map(marital_bin_mapping)

Confirm the changes. Check the new columns values and distribution.

In [45]:

print(data['Marital Status Bin'].value_counts(normalize=True, ascending=False).mul(100).round(2))
display(data.head())

Marital Status Bin
1    87.16
2     9.49
4     2.35
7     1.00
Name: proportion, dtype: float64

	Marital status	Application mode	Application order	Course	Daytime/evening attendance\t	Previous qualification (grade)	Nationality	Mothers occupation	Fathers occupation	Admission grade	Displaced	Debtor	Tuition fees up to date	Gender	Scholarship holder	Age at enrollment	Curricular units 1st sem (enrolled)	Curricular units 1st sem (evaluations)	Curricular units 1st sem (approved)	Curricular units 1st sem (grade)	Curricular units 2nd sem (enrolled)	Curricular units 2nd sem (evaluations)	Curricular units 2nd sem (approved)	Curricular units 2nd sem (grade)	Curricular units 2nd sem (without evaluations)	Unemployment rate	Inflation rate	GDP	Previous qualification Bin	Mothers qualification Bin	Fathers qualification Bin	Marital Status Bin
1	1	15	1	9254	1	160.0	1	3	3	142.5	1	0	0	1	0	19	6	6	6	14.000000	6	6	6	13.666667	0	13.9	-0.3	0.79	7	7	9	1
3	1	17	2	9773	1	122.0	1	5	3	119.6	1	0	1	0	0	20	6	8	6	13.428571	6	10	5	12.400000	0	9.4	-0.8	-3.12	7	5	3	1
4	2	39	1	8014	0	100.0	1	9	9	141.5	0	0	1	0	0	45	6	9	5	12.333333	6	6	6	13.000000	0	13.9	-0.3	0.79	7	3	5	2
5	2	39	1	9991	0	133.1	1	9	7	114.8	0	1	1	1	0	50	5	10	5	11.857143	5	17	5	11.500000	5	16.2	0.3	-0.92	6	3	3	2
6	1	1	1	9500	1	142.0	1	7	10	128.4	1	0	1	0	1	18	7	9	7	13.300000	8	8	8	14.345000	0	15.5	2.8	-4.06	7	6	5	1

We see that the values for Facto Union, Legally Separated, Widower are now all under 7 in the new Marital Status Bin column. Let's go ahead and delete the original Marital status column and view the dataframe to confirm.

In [46]:

data.drop('Marital status', axis=1, inplace=True)

Have a look at the data to check once again.

In [47]:

display(data.head(15))

	Application mode	Application order	Course	Daytime/evening attendance\t	Previous qualification (grade)	Nationality	Mothers occupation	Fathers occupation	Admission grade	Displaced	Debtor	Tuition fees up to date	Gender	Scholarship holder	Age at enrollment	International	Curricular units 1st sem (enrolled)	Curricular units 1st sem (evaluations)	Curricular units 1st sem (approved)	Curricular units 1st sem (grade)	Curricular units 1st sem (without evaluations)	Curricular units 2nd sem (enrolled)	Curricular units 2nd sem (evaluations)	Curricular units 2nd sem (approved)	Curricular units 2nd sem (grade)	Curricular units 2nd sem (without evaluations)	Unemployment rate	Inflation rate	GDP	Previous qualification Bin	Mothers qualification Bin	Fathers qualification Bin	Marital Status Bin
1	15	1	9254	1	160.0	1	3	3	142.5	1	0	0	1	0	19	0	6	6	6	14.000000	0	6	6	6	13.666667	0	13.9	-0.3	0.79	7	7	9	1
3	17	2	9773	1	122.0	1	5	3	119.6	1	0	1	0	0	20	0	6	8	6	13.428571	0	6	10	5	12.400000	0	9.4	-0.8	-3.12	7	5	3	1
4	39	1	8014	0	100.0	1	9	9	141.5	0	0	1	0	0	45	0	6	9	5	12.333333	0	6	6	6	13.000000	0	13.9	-0.3	0.79	7	3	5	2
5	39	1	9991	0	133.1	1	9	7	114.8	0	1	1	1	0	50	0	5	10	5	11.857143	0	5	17	5	11.500000	5	16.2	0.3	-0.92	6	3	3	2
6	1	1	9500	1	142.0	1	7	10	128.4	1	0	1	0	1	18	0	7	9	7	13.300000	0	8	8	8	14.345000	0	15.5	2.8	-4.06	7	6	5	1
8	1	3	9238	1	137.0	62	9	9	129.3	0	0	1	0	1	21	1	6	8	6	13.875000	0	6	7	6	14.142857	0	16.2	0.3	-0.92	7	7	7	1
10	1	1	9670	1	139.0	1	5	7	130.6	1	0	1	0	0	18	0	6	6	6	12.333333	0	6	7	5	14.200000	0	13.9	-0.3	0.79	7	5	6	1
11	1	1	9500	1	136.0	1	9	9	119.3	1	0	1	0	1	18	0	8	8	7	13.214286	0	8	8	7	13.214286	0	12.7	3.7	-1.70	7	6	5	1
13	53	1	9254	1	110.0	1	4	7	111.8	1	0	1	0	1	21	0	6	7	6	10.571429	0	6	8	5	11.000000	0	8.9	1.4	3.51	9	7	7	1
14	1	1	9085	1	149.0	1	5	5	137.1	1	0	1	0	1	18	0	5	7	4	13.250000	0	5	5	5	12.000000	0	10.8	1.4	1.74	7	5	3	1
16	18	1	9238	1	137.0	1	5	8	137.4	1	0	1	0	0	18	0	6	10	1	12.000000	0	6	14	2	11.000000	0	10.8	1.4	1.74	7	6	5	1
17	17	2	9500	1	135.0	1	5	4	127.3	1	0	1	0	0	18	0	7	8	7	13.306250	0	8	8	8	14.545000	0	15.5	2.8	-4.06	7	6	7	1
18	1	1	9130	1	137.0	1	3	5	136.3	1	0	1	0	0	20	0	5	8	4	12.500000	1	5	8	4	12.250000	2	10.8	1.4	1.74	7	9	6	1
19	1	1	9853	1	140.0	1	7	7	124.6	1	0	1	0	0	18	0	7	7	6	11.666667	0	7	8	6	13.500000	0	16.2	0.3	-0.92	7	6	6	1
20	1	3	171	1	122.0	1	9	8	120.3	0	0	1	0	1	21	0	0	0	0	0.000000	0	0	0	0	0.000000	0	11.1	0.6	2.02	7	7	7	1

Great. No more Marital status column. It has been replaced with Marital Status Bin that has only 4 category values.

Let's look at what else is left. Below is the same list I created earlier above after looking through the data and seeing the high cardinality categorical columns that need decreasing.

Application mode
Application order
Course
Fathers occupation
Mothers occupation
Nationality
~~Marital status~~
~~Previous qualification~~
~~Fathers qualification~~
~~Mothers qualification~~

Onto the Nationality column now.

Binning Nationality Variable

Look again at the distribution of the Nationality column using .value_counts descending from most to least frequency.

Warning:

Output truncated. This notebook contains too many cells to display efficiently.

4.0 MiB Raw Blame History