# Evaluation Framework¶

**SDV** contains a *Synthetic Data Evaluation Framework* that facilitates
the task of evaluating the quality of your *Synthetic Dataset* by
applying multiple *Synthetic Data Metrics* on it and reporting results
in a comprehensive way.

## Using the SDV Evaluation Framework¶

To evaluate the quality of synthetic data we essentially need two things:
*real* data and *synthetic* data that pretends to resemble it.

Let us start by loading a demo table and generate a synthetic replica of
it using the `GaussianCopula`

model.

```
In [1]: from sdv.demo import load_tabular_demo
In [2]: from sdv.tabular import GaussianCopula
In [3]: real_data = load_tabular_demo('student_placements')
In [4]: model = GaussianCopula()
In [5]: model.fit(real_data)
In [6]: synthetic_data = model.sample()
```

After the previous steps we will have two tables:

`real_data`

: A table containing data about student placements

```
In [7]: real_data.head()
Out[7]:
student_id gender second_perc high_perc high_spec degree_perc degree_type work_experience experience_years employability_perc mba_spec mba_perc salary placed start_date end_date duration
0 17264 M 67.00 91.00 Commerce 58.00 Sci&Tech False 0 55.0 Mkt&HR 58.80 27000.0 True 2020-07-23 2020-10-12 3.0
1 17265 M 79.33 78.33 Science 77.48 Sci&Tech True 1 86.5 Mkt&Fin 66.28 20000.0 True 2020-01-11 2020-04-09 3.0
2 17266 M 65.00 68.00 Arts 64.00 Comm&Mgmt False 0 75.0 Mkt&Fin 57.80 25000.0 True 2020-01-26 2020-07-13 6.0
3 17267 M 56.00 52.00 Science 52.00 Sci&Tech False 0 66.0 Mkt&HR 59.43 NaN False NaT NaT NaN
4 17268 M 85.80 73.60 Commerce 73.30 Comm&Mgmt False 0 96.8 Mkt&Fin 55.50 42500.0 True 2020-07-04 2020-09-27 3.0
```

`synthetic_data`

: A synthetically generated table that contains data in the same format and with similar statistical properties as the`real_data`

.

```
In [8]: synthetic_data.head()
Out[8]:
student_id gender second_perc high_perc high_spec degree_perc degree_type work_experience experience_years employability_perc mba_spec mba_perc salary placed start_date end_date duration
0 17313 M 41.068012 56.085381 Commerce 44.883622 Comm&Mgmt False 1 50.014502 Mkt&HR 50.329875 24956.891559 True 2020-04-05 2021-04-11 12.0
1 17459 M 83.551016 65.519349 Commerce 67.995809 Sci&Tech False 1 77.288552 Mkt&Fin 66.383096 31318.783931 True 2020-01-25 2020-07-03 NaN
2 17459 F 71.216682 68.833595 Commerce 85.226367 Comm&Mgmt False 0 54.276205 Mkt&Fin 67.559014 25514.777281 True 2020-01-08 2020-02-05 3.0
3 17298 M 80.828939 76.962830 Commerce 73.777104 Comm&Mgmt False 1 75.191165 Mkt&Fin 78.558689 35690.903886 True 2020-01-11 2020-06-09 3.0
4 17440 F 63.275924 62.635678 Commerce 61.609046 Comm&Mgmt False 1 65.422342 Mkt&Fin 71.237561 32701.627256 True 2020-01-06 2020-01-06 3.0
```

Note

For more details about this process, please visit the GaussianCopula Model guide.

### Computing an overall score¶

The simplest way to see how similar the two tables are is to import the
`sdv.evaluation.evaluate`

function and run it passing both the
`synthetic_data`

and the `real_data`

tables.

```
In [9]: from sdv.evaluation import evaluate
In [10]: evaluate(synthetic_data, real_data)
Out[10]: 0.6093353724148053
```

The output of this function call will be a number between 0 and 1 that will indicate how similar the two tables are, being 0 the worst and 1 the best possible score.

### How was the obtained score computed?¶

The `evaluate`

function applies a collection of pre-configured metric
functions and returns the average of the scores that the data obtained
on each one of them. In most scenarios this can be enough to get an idea
about the similarity of the two tables, but you might want to explore
the metrics in more detail.

In order to see the different metrics that were applied you can pass and
additional argument `aggregate=False`

, which will make the
`evaluate`

function return a dictionary with the scores that each one
of the metrics functions returned:

```
In [11]: evaluate(synthetic_data, real_data, aggregate=False)
Out[11]:
metric name raw_score normalized_score min_value max_value goal
1 LogisticDetection LogisticRegression Detection 0.339336 3.393359e-01 0.0 1.0 MAXIMIZE
2 SVCDetection SVC Detection 0.406922 4.069220e-01 0.0 1.0 MAXIMIZE
11 GMLogLikelihood GaussianMixture Log Likelihood -42.161881 4.890215e-19 -inf inf MAXIMIZE
12 CSTest Chi-Squared 0.896715 8.967145e-01 0.0 1.0 MAXIMIZE
13 KSTest Inverted Kolmogorov-Smirnov D statistic 0.928488 9.284884e-01 0.0 1.0 MAXIMIZE
14 KSTestExtended Inverted Kolmogorov-Smirnov D statistic 0.921395 9.213953e-01 0.0 1.0 MAXIMIZE
27 ContinuousKLDivergence Continuous Kullback–Leibler Divergence 0.581344 5.813441e-01 0.0 1.0 MAXIMIZE
28 DiscreteKLDivergence Discrete Kullback–Leibler Divergence 0.828046 8.280459e-01 0.0 1.0 MAXIMIZE
```

### Can I control which metrics are applied?¶

By default, the `evaluate`

function will apply all the metrics that
are included within the SDV Evaluation framework. However, the list of
metrics that are applied can be controlled by passing a list with the
names of the metrics that you want to apply.

For example, if you were interested on obtaining only the `CSTest`

and
`KSTest`

metrics you can call the `evaluate`

function as follows:

```
In [12]: evaluate(synthetic_data, real_data, metrics=['CSTest', 'KSTest'])
Out[12]: 0.9126014582897475
```

Or, if we want to see the scores separately:

```
In [13]: evaluate(synthetic_data, real_data, metrics=['CSTest', 'KSTest'], aggregate=False)
Out[13]:
metric name raw_score normalized_score min_value max_value goal
0 CSTest Chi-Squared 0.896715 0.896715 0.0 1.0 MAXIMIZE
1 KSTest Inverted Kolmogorov-Smirnov D statistic 0.928488 0.928488 0.0 1.0 MAXIMIZE
```

For more details about all the metrics that exist for the different data modalities please check the corresponding guides.