Understanding Daily Solar Energy Consumption

Creating a daily report to help me understand my consumption of solar electricity behind the meter.

The Challenge

For the last few years I've had an itch I wanted to scratch around understanding how much of the solar electricity I generate that I consume before it is exported to the grid. It's a relatively tricky number to get at, but it's only by understanding the cumulative total of how much solar I consume AND how much grid electrify I consume, that I can understand how much I truly use.

By understanding my solar and grid consumption and how much I export, I can better plan ahead for my next solar system or even calculate the return on investment of battery storage. More importantly, as a family, we can look at ways to modify our habits to take full advantage of the solar we generate and reduce our consumption from the grid.

I've written this post to help people understand the various options available to them to understand their electricity consumption and also show how I've managed to scratch my itch.

What am I trying to do?

On a daily basis I want to understand my total power consumption including:

• Power consumed from the grid
• Power consumed directly from my solar panels before it is sent to the grid
• Power exported to the grid.

This will let me calculate:

• Approximate cost of electricity sourced from the grid
• Savings made by using solar before it hits the grid
• Approximate number of kW I'd be able to use to fill a battery (and therefore save myself from buying from the grid).

If I store this daily information in a database, I'd also be able to give myself weekly/monthly/yearly reports to even out the peaks and troughs of weather and usage and get a really clear idea on the payback of getting a battery (which will be a fair bit cheaper if Labor's latest battery-election-win promise happens after our next election).

Lets break down the pieces of the puzzle.

Understanding electricity consumption from the grid and solar feed-in to the grid

On a monthly or quarterly basis, it's easy for most Australians to understand how much they consume from the grid because they pay a power bill that tells them.

If you're with a modern power retailer like Powershop, you'll be able to easily access usage and export data down to 30 minute chunks using their mobile apps or web portal. Powershop used to have a public API, but it seems to have been discontinued or at least made private.

Older, more traditional retailers will only give you a monthly total on your bill. Often if you ask for it they can manual extract to a CSV of your usage data, but getting this CSV requires a manual process or phone call. There are better options available as my experience is that this is like extracting blood from a stone (Red Energy tried to charge me $15 for a CSV of my usage, so I left them for Powershop).

If you have Powershop, you can get at your data lots of ways. For the purposes of this experiment, I've used the web portal as opposed to the app, as it'll let me extract the data in a few ways later on to get that elusive solar consumption number.

Your daily usage data, broken into 30 minute chunks is useful for determining when you're using the most/least electricity. The black areas in the middle of some days definitely show that my solar is kicking in and preventing a chunk of grid usage.

The other ways I can get at my data are by downloading a CSV file of my usage. The CSVs provided by Powershop are far easier to understand than the ones you can get from your utility.

In my case, a typical usage CSV from Powershop is pretty easy to understand.

Another way to get at your usage data is through your Distributors portal (the companies who make electricity are known as Generators, the companies who manage the wires and poles in your area are known as Distributors and the people you buy it of are called Retailers). In my area the distributor is PowerCor and they have a tool called the MyEnergy portal. You can usually access your data using your meter's NMI (National Meter Identifier), which is found on your bill.

Logging in to your distribution portal will usually give you even more information than you'd get from a retailer as they have access to all the data from a region (many retailers can operate in the same distribution area). In particular things like Suburb average usage data is something they can tell you.

Average usage data is useful to be able to tell if you're using more or less than others in your area, but it obviously doesn't account for your particular families usage habits (eg. people at home during the day, people away during the day etc). It also doesn't account for behind the meter solar usage so it may not be accurate.

Using Powercor, you can either see the information you want on their dashboard, or you can download a more detailed CSV format that gives you historical import/export data.

Beware that the formats that the data comes in are particularly complex and difficult to understand. If you have the choice of downloading through Powercor or Powershop, I'd suggest that the Powershop format is preferential.

PowerCor CSVs aren't super easy to understand. I have no idea what I'm looking at.

If anyone from Powercor is reading this, it would be awesome to have a few more spreadsheet options for 5 minute or 30 minute increment data with each day having three rows showing:

1. Grid consumption
2. Solar feed-in
3. Suburb average consumption
4. Suburb average feed-in
5. That would blow my mind. Anyway.

Understanding solar usage in your home before the grid.

As I've already said, if you have solar panels, it's likely that you have some understanding how much you send back to the grid through your powerbill as you should receive credit for the solar energy you feed back into the grid.

The problem is that there is a delta between how much solar you generate via your rooftop solar panels, and how many kW of electricity you receive from your retailer.

The difference exists because, hopefully often, your household is using electricity at the same time that it is being generated. The solar industry calls this "behind the meter" consumption. This means that it's consumed by your appliances directly, rather than being exported to the grid. Without a special addition to your smart meter - like a Fronius Smart Meter - this number can be difficult to understand as it relies on being able to compare consumption and export data for the same time period from your bills and inverter.

Inverters with a locally hosted dashboard.

Almost all of the inverters I've seen in the last 5 or so years have a web portal you can access either via your local home network. This is the portal I have access to via my 2kW Enasolar inverter.

Inverters sending their generation data to a cloud based dashboard.

Most modern inverters also store your generation data in the cloud and make it visible via a web portal, which is ever better.

Interestingly, when I purchased my 2kW Enasolar inverter it came with a cloud-hosted portal I could login to and see my usage data remotely. Since I bought it though that portal seems to have been decommissioned.

The web portals available these days seem to be getting much better. Fronius' solar.web is excellent and even comes with apps for your Apple TV, iPhone or Apple Watch. Super cool.

Understanding behind-the-meter solar usage

Now that we know we have access to data about how much we buy from the grid, sell to the grid and generate, all on a daily basis, we can do some maths.

Solar Generated - Solar Exported to the Grid + Power Consumed from the Grid = Total power consumption

But how to get at this info automatically on a daily basis without access to APIs? In my situation I'm faced with the following set up:

1. Inverter is only available via my local network (it doesn't have a cloud portal). My inverter can give me the Solar Generated I seek
2. My Powershop account is available via the web or iOS app. It can tell me the Solar Exported to the Grid and the Power Consumed from the Grid amounts.

Neither of these services have APIs I am aware of, but I can get at both through my web browser if I'm at home.

Enter selenium web driver

At work we've got lots of automated testing and I've always loved the way that you can drive a web browser via code instead of a mouse.

I also had an old 11" Macbook Air lying in a cupboard.

Putting those two pieces together I've [written a little ruby script] that drives the MBA to.

1. Extract meter data from Powershop

Meter data is available from Powershop the next day at about 6 - 6:30am. So at 7am, my Selenium script runs an

1. Visits Powershop, logs in.
2. Visits the meter readings page.
3. Extracts the last two whole days worth of import and export readings (subtracting the day-before-yesterday's reading from yesterdays reading will tell you the kWh used - you can verify this in the Powershop app).
4. This gives us some variables:
   • @solarExportReadingYesterday
   • @solarExportReadingDayBeforeYesterday
   • @powerImportReadingYesterday
   • @powerImportReadingDayBeforeYesterday
5. Extract data from my inverter
6. Visit my inverters page on the local network
7. Find the element on the page telling me yesterdays generation
8. Save it into the variable @solarGeneratedYesterday.

Put the pieces together

With these 5 variables, I can now do some calculations to figure out a whole lot.

How much solar did I feed into to the grid (and how much did I make for that @ 11.17c per kWh)

solarFeedIn = (@solarExportReadingYesterday.to_f - @solarExportReadingDayBeforeYesterday.to_f).round(2)solarFeedInRebate = ((solarFeedIn.to_f * 11.17)/100).round(2)

How much power did I purchase from the grid (and how much did that cost me @ 30.14c per kWh?)

powerPurchase = (@powerImportReadingYesterday.to_f - @powerImportReadingDayBeforeYesterday.to_f).round(2)powerPurchaseCost = ((powerPurchase.to_f * 30.14)/100).round(2)

How much solar did I consume (and how much did this save me by not having to buy it @ 30.14c per kWh)

solarConsumption = (@solarGeneratedYesterday.to_f - solarFeedIn.to_f).round(2)solarConsumptionSavings = ((solarConsumption.to_f * 30.14)/100).round(2)

And, most awesomely, when you combine my solar usage and power purchase, you get how much electricity I use in my home.

totalConsumption = (solarConsumption + powerPurchase).round(2)

I can also calculate how much my electricity cost, factoring in credits I receive for exported power back to the grid.

overallCost = (powerPurchaseCost - solarFeedInRebate).round(2)

Calculating the payback of a battery

One of the other things I can do with this data, if a little crudely, is to calculate how much I'd have saved if, instead of sending my power to the grid, I'd stored it in a battery and used it to reduce my consumption of grid power. This is a crude number because it's not always clear that I'd be able to store or use the power stored in my battery, but so far my usage patterns check out. I would easily be able to use all the energy we feedback to the grid if it were available at other times during the day. The question then becomes whether, over a longer period, it's financially viable (a ~14kWh battery costs about $10k and is warranted for about 10 years at the moment).

potentialBatterySavings = ((solarFeedIn.to_f * 30.14)/100).round(2)potentialBatteryModifiedGridUsage = (powerPurchase - solarFeedIn).round(2)

Calculating the payback period of solar panels

Having this data will also let me calculate the payback period for my current solar array more precisely. It'll also let me calculate how big an array I'd need if we were able to get more panels if I wanted to fully offset my grid consumption.

The daily report

Data that isn't used is pretty useless. To make sure I didn't fall into this trap, the last step in my script is to send a push notification to my phone. The little report I get each day is packed full of info and is quite useful.

Try it out

I'd fully recommend understanding your energy usage, especially now given the increasing affordability of battery storage. Do you have enough solar panels? Are you using power at the right time of day? All these questions can be answered by understanding your usage more comprehensively.

To give my script a go, you'll need:

• Solar panels with a web or local network connected inverter
• Powershop (swapping is easy - retailers don't traditionally reward loyalty, so consider swapping - use my referral code for an extra $75)
• An old Mac you can use to run the script (you could probably get most of this working on a newer raspberry pi. My Model B couldn't cut it)
• An account with an online service called Pushover (to send Push notifications - you could just as easily swap this for a tool like Mailgun)
• I use the app LingonX to schedule the command to run on my Mac via Launchd. It's been a good investment for me as I use it across quite a few projects to automate various things.

Download the scripts

Thanks for reading. I hope you found this useful.