Digital Refinery Specialist

Manipulating and visualizing data

Variables and assignment

• Introduction to Python command line.
• Basic assignment syntax and dynamic typing.
• Asking for help. Finding help online.
• Native data types
• int and float (and complex); why so many kinds of numbers?
• str.
• Type casting.
• String methods, strings as collections. Indexing and slicing.
• String formatting anf f-strings.
• Exercise: text processing practice manipulating formation names.

Operators and expressions

• Mathematical operations, comparison operators, booleans.
• Augmented assignment, multiple assignment.
• Copies and pointers.
•  Data collections and data structures
• list — indexing again, slicing again, striding, nested lists.
• Exercise:

Flow control

• Iteration and iterables: for and while.
• Exercise: compute impedance and reflectivity in a well.
• List comprehensions.
• Exercise: convert for loops into list comprehensions.
• Making decisions: if-else statements.
• Exercise:

Getting data, part 1

• Reading and writing text files.
• Exercise: create a dictionary of well tops from a ‘broken’ input text file.
• Functions
• Built-in functions, and importing modules.
• The anatomy of a function. Syntax, docstrings.
• Exercise: write a function that computes an impedance log.
• Exercise: write a function that computes reflection coefficients.
• Exercise: write a function that computes formation thicknesses.
• Sharing code via modules, importing and using modules.
• Exercise: Getting data from a sidewall core analysis report (csv file).
• Exercise: Get geological ages by processing pages.
• The Python standard library.
• External Python packages and PyPi.

NumPy

• What is NumPy for? n-dimensional array
• Exercise: 1
• Exercise: 2

matplotlib

• What is matplotlib?
• Exercise: Exploring plots.
• Seaborn, Plotly, Bokeh, and other plotting environments.

SciPy

• What's in the SciPy package?
• Exercise: make an offset synthetic seismogram.
• Spectral analysis with scipy.
• Exercise: make a time-frequency plot of our synthetic.

Pandas, a quick introduction

• What is pandas? When do we use pandas vs ndarrays?
• Exercise: Loading and cleaning a dataset with pandas.
• Exercise: A quick introduction to GeoPandas.

Reading and writing data files

• Persisting ndarrays, dataframes, and other objects. Pickling.
• Reading SEG-Y files with ObsPy. Writing SEG-Y files.
• Reading LAS files with lasio and welly. Writing LAS files.
• Reading and writing SHP files with fiona.

Getting data, part 2

• Databases. SQL vs NoSQL. Libraries for hitting databases.
• Exercise: Storing objects in a database, and retrieving them again. Querying.
• Exercise

Writing code

• Text editors, IDES, Jupyter.
• Linting and PEP8.
• Documentation, testing, continuous integration.

Classes and object-oriented programming

• Everything is an object.
• Why use classes?
• Exercise:

Version control

• Introduction to git and GitHub.
• Exercise:

Runtime

• Conda environments. Other options.
• Exercise: build and clone a conda environment for your project.
• Containers, Docker, and developer operations.

Test driven development

• Untested code is broken code.
• Writing tests.

Writing code

• Text editors, IDES, Jupyter.
• Linting and PEP8.
• Documentation, testing, continuous integration.

Classes and object-oriented programming

• Everything is an object.
• Why use classes?
• Exercise:

Version control

•  Introduction to git and GitHub.
• Exercise:

Runtime

• Conda environments. Other options.
• Exercise: build and clone a conda environment for your project.
• Containers, Docker, and developer operations.

Test driven development

• Untested code is broken code.
• Writing tests.
• Exercise: write the first tests for our well log class.

Documentation

• Writing self-documenting code: docstrings and comments.
• Supporting documents and notebooks.
• Exercise:

Packaging

• Functions, files, modules, and packages — review.
• Setup.py, requirements.txt, PyPi, and everything else.
• Managing branches in git.

Getting better

•  Tips for becoming a better programmer.
• Online resources. Conferences and meetings.

Introduction

• Recognizing tasks suitable for machine learning.
• What's the difference between supervised and unsupervised learning?
• Recognizing regression vs classification tasks.

Data management for machine learning

• DataFrames: A new way to look at well logs.
• Exercise: loading a pandas DataFrame from a CSV.
• Exercise: building a pandas DataFrame
• DataFrames vs arrays (vs Hadoop, Dask, etc).

The machine learning iterative loop

• Data — Getting the data. Loading and storing in an array and/or DataFrame
• Processing — data exploration, inspection, cleaning, and feature engineering.
• Model — What is a model? Training a Scikit-Learn model (for now).
• Results — assessing quality and performance metrics (accuracy, recall, F1, confusion matrices)

• Build two use cases from scratch till final visualisations