Data Miner

Data mining combines statistics, artificial intelligence (AI) and database research. Data miners focus on identifying predictors of relevant or predetermined outcomes.

There are three stages in data mining:

• initial exploration (for example, generate a summary and description of the key variables and values)
• model building or pattern identification
• model testing (for example, applying the model to new data to generate predictions).

In general data miners:

• clean the data (remove data that may have errors or contradict other data in the pattern)
• select relevant subsets of records to reduce the amount of data to explore
• identify the most relevant variables
• determine the complexity and nature of models required
• use techniques such as bagging (averaging), stacking, bootstrapping and meta-learning
• apply different models (for example, association mining, classification mining, clustering mining, neural networks, Bayesian networks) to the same data and compare results to select a preferred model
• apply the selected model to new data to find patterns and clusters or test and generate predictions or estimates of expected outcomes.

Some data miners may use prescriptive analytics (building different prediction models based on various potential decision actions) to guide decision makers to a desired outcome.

Data miners work in offices with sophisticated information technology. They usually work standard weekday office hours but may be required to work overtime to meet project deadlines.

Data miners often work on several different projects at the same time. They generally work in a team environment but may work alone.

Data miners need the following characteristics:

• an aptitude for and interest in mathematics, statistics and databases
• an interest in applying scientific principles to solve practical problems
• the ability to analyze information and understand abstract relationships
• the ability to think logically, organize projects and carry them out
• the ability to work alone as well as with others
• the ability to communicate with individuals from different technical backgrounds
• strong attention to detail.

Data miners should enjoy synthesizing data, applying statistical theories and methods, and working with others to understand and solve problems.

Emerging occupations typically are the result of:

• an increased human need (for example, alternate sources of energy)
• technological advances
• greater specialization within an occupation.

Often there are too few people working in an emerging occupation to gather survey information. Therefore, it can be difficult to define advancement opportunities or employment outlook. Some Albertans already are working in this emerging occupation but future demand for data miners is unknown.

Data miners work for a variety of employers including:

• medical and educational research agencies
• natural resource companies
• environmental research companies
• market research companies
• pharmaceutical companies
• large retail companies
• post-secondary schools.

Most jobs are located in major urban areas.

Outstanding individuals who have graduate degrees and many years of experience may move into management positions or become private consultants.

The employment outlook in this occupation will be influenced by a wide variety of factors including:

• trends and events affecting overall employment (especially in the industries listed above)
• location in Alberta
• employment turnover (work opportunities generated by people leaving existing positions)
• occupational growth (work opportunities resulting from the creation of new positions that never existed before)
• size of the occupation.