The electricity industry in most of Texas is deregulated, which means that consumers can choose their electricity provider. This is different from regulated areas of Texas and other parts of the country, where a single utility company delivers electricity to homes and businesses. Texas deregulation process started with the passage of Senate Bill 373 in 1995 and culminated with granting independence to the Electric Reliability Council of Texas (ERCOT) in 2002.
My work at a green energy trading company/hedge fund (battery energy storage system or "BESS" company) involved building an entire backend real-time, multi-cloud data storage system and ELT/ETL pipeline. Our goal was to make real-time energy predictions as accurately as possible and as far in advance as possible.
I cannot provide specifics on the data vendors or code used to build my machine learning models out of confidentiality, but here is an example of the type of weather data that such models use:
Looks cool, right?
It was a fun challenge to work with both public and private, proprietary data sets to build a data pipeline on Snowflake, AWS and with an app on Slack.
The tangible outputs of weather are simple and easy to grasp: sunshine, rain, etc. But this becomes a more complex topic when considering that solar irradiance, wind velocity and direction, precipitation, temperature, and many more variables vary by location and over time. This is an especially interesting problem because these weather patterns have an impact on the physical capacity of the power lines that transmit electricity, and then there is the idea of these power lines have limited capacity much like highways, areas of higher demand at different times and how that varies by location. There is also the interesting consideration that solar power generation and wind generation obviously varies with the day-night cycle, and is also impacted by cloud cover.
Building the model was the fun part.
A brief article that I wrote about my work can be read on this industry publication: https://www.energy-storage.news/the-future-of-battery-trading-addressing-the-risks-and-rewards-of-ai-optimisation/