Transformation of data

Transformations are widely used in statistics to reduce data to standard forms. Some common methods of re-expressing data are as follows:

• Centering The sample mean (column mean) $\overline{x}$ is subtracted from the data values x_i in order to obtain centered ``anomalies'' $x_i-\overline{x}$ having zero mean. All information about mean location is lost.

• Standardizing The data values are centered and then divided by their standard deviations to obtain ``normalised anomalies'' (meteorological notation) having zero mean and unit variance. All knowledge of location and scale is lost and so statistics based on standardised anomalies are unaffected by any shifts or rescaling of the original data. Standardizing makes the data dimensionless and so is useful for defining standard indices. Also note that correlation coefficients are unaffected by any linear transformations such as standardization.

• Normalizing

  Normalizing transformations are non-linear transformations often used by statisticians to make data more normal (Gaussian). This can reduce bias caused by outliers, and can also transform data to satisfy normality assumptions that are assumed by many statisitical techniques (e.g. power spectra). Note: meteorologists often say ``normalizing'' when they should say ``standardizing'' (not recommended !).

  A much used class of transformations is the Box-Cox power law transformation $y=(x^\lambda-1)/\lambda$, where $\lambda$ can be optimally tuned from 0 to 1. In the limit as $\lambda\rightarrow 0$, one obtains the $y=\log x$ transformation much used to make postively skewed quantities such as stockmarket prices more normal. The $\lambda=0.5$ square root transformation is often a good compromise for postively skewed variables such as rainfall amounts (Stephenson et al. 1999).