Category: data

01 May

Exploring Citywide Energy Usage Part II

Sarah Newman benchmarking, data, energy, LL84 0 Comments

HOW HAS CITYWIDE MULTIFAMILY ENERGY CONSUMPTION CHANGED?

In a recent blog post, we explored how energy consumption varies geographically across New York City using the publicly disclosed Local Law 84 data from 2015. In this follow-up, we look at how energy use has changed in the years that the benchmarking law has been in place. Using publicly disclosed data for the last four years and data provided by the NYC Office of Sustainability for the first two, we are able to assess how the law continues to shape New York City buildings for the better.

Data Cleaning Methodology

In this analysis, we use weather-normalized Source EUI1 as calculated by Energy Star’s Portfolio Manager software, which will just be referred to as “EUI” in the remainder of the blog. Given that the first year’s data (2010) reduces the overall dataset size by over half and is the most suspect in terms of data quality, we opt to instead include only the last five years of energy disclosure data. In addition, we require that buildings have a valid EUI for each of the last five years and remove buildings that vary in their reported square footage by more than 10% between any two years or in their EUI by more than 60%, either of which would indicate that a data entry error is likely. We also remove the top and bottom 1% of EUIs from all years pooled together. This leaves a total of 2,282 multifamily buildings in our dataset.

Change in Median Source EUI

After cleaning the data, we analyze the change in the median EUI over the last five years (see figure below), fitting it with an ordinary least squares regression. The best fit model implies a decrease in the median EUI of 1.3 kBTU/sqft/yr each year (or 1%/year), with a total decrease in EUI of 5.2 kBTU/sqft/yr over the full 5 years. This linear model has an adjusted r² value of 0.75, indicating a robust fit.

This change in the median EUI can be thought of as how much the average building population is changing its consumption overall. We can also look at the median change in EUI from 2011 to 2015 (6.0 kBTU/sqft/yr or 4.6% over the five-year span), which is more representative of how a typical building has changed over those five years.

Table 1 shows the median EUI for each year. It is important to note that these values vary somewhat from those published in NYC’s Benchmarking reports as the former are derived only from buildings with valid data in all five years.

Table 1. Median Source EUI from 2011-2015

    Year     Median Source
EUI (kBTU/sqft/yr)
2011 129.9
2012 126.3
2013 125.8
2014 124.2
2015 124.4

Looking Across All Six Years

If we instead include all six years of data, the decrease in median EUI is a bit larger (1.7 kBTU/sqft/yr/yr), but as noted before, this reduces the overall dataset size by more than half to only 808 buildings. The histogram below shows the distribution of 2015 EUIs for three different data subsets, as well as the median for each subset (vertical dashed lines):

  • Buildings with data in years 2012-2015
  • Buildings with data in years 2011-2015 (the dataset used throughout this blog)
  • Buildings with data in all 6 years

While there is not a large difference in the total number or distribution of EUIs for the first two groups, the subset including only buildings with data in all six years has a much higher median EUI for the overlapping five years (127.8 vs 124.4 for 2015), meaning that buildings that complied in the first year have on average higher EUIs than those that didn’t.

Comparing the Change for the Best and Worst Buildings

We can also see the shift in the EUI over the last five years by looking at the change in the normalized EUI distribution with time. The figure below displays the normalized EUI distribution (from a Kernel-density estimate) for each year, showing that the EUI distribution becomes narrower with a lower median EUI over time.

In order to confirm that this trend is not just year-to-year noise in building consumption, we want to see that the worst buildings are improving significantly and the best buildings are not getting worse. We can see this is the case in Table 2, which shows the median 2011 and 2015 EUIs of buildings in the 1st, 2nd, 3rd and 4th quartiles of buildings from the 2011 data.

Table 2. Change in Median Source EUI for 2011 Quartiles

2011  Quartiles 

2011 Median Source EUI (kBTU/sqft/yr) 2015 Median Source EUI (kBTU/sqft/yr) 2011 to 2015 EUI % Difference

1st

98.3 100.6

2%

2nd

120.5 115.6

-4%

3rd

140.7

132.4

-6%

4th 172.7 155.8

-10%

While the buildings in the best quartile have increased their usage by 2%, the remaining 3/4 of buildings have improved, with the worst quartile of buildings decreasing their usage by 10% – suggesting that overall, buildings in NYC are improving.

This reduction in energy consumption can also be evaluated with a paired t-test, comparing the distributions of EUIs in 2011 and 2015, which returns a p-value of 6×10−28, indicating that the distributions of EUIs in those two years are statistically distinct.

Excluding the Impact of Hurricane Sandy

Thus far in this analysis, we have not compensated for the effects of Hurricane Sandy which cut out power to many areas in NYC for weeks in 2012. In the figure below, we remove areas with prolonged outages (Red Hook, The Rockaways and Lower Manhattan) from the dataset and compare the EUI trend between that set and our main dataset.

The ‘Sandy-corrected’ group, which omits heavily affected areas, shows the same trend of decreasing EUI over time, albeit with an even steeper slope (1.45 kBTU/sqft/yr vs 1.3 kBTU/sqft/yr). The median EUI for each year is also slightly lower for the Sandy-corrected group (see Table 3), likely because buildings in lower Manhattan have relatively high EUIs (see interactive map in the following section).

Table 3. Median Source EUI for Sandy-corrected Data

  Year   

Median Source EUI
(kBTU/sqft/yr)
Median Source EUI
Sandy-corrected (kBTU/sqft/yr)

2011

129.9 128.7

2012

126.3

125.2

2013 125.8

124.5

2014 124.2

122.7

2015

124.4

122.7

ARE ALL PARTS OF NYC IMPROVING?

We can also explore how this yearly change in EUI varies geographically. Are all boroughs and neighborhoods in NYC improving at the same pace? Are some still getting worse?

2011-2015 Median Source EUI by Community District

The maps below show the median EUI for each NYC community district for each year from 2011 to 2015 as well as the absolute and % difference from 2011 to 2015. You can click on the tabs at the top of the map to switch years, and hover over a community district to see the median EUI and number of buildings.

From these maps we can see that energy consumption has decreased in most regions, and in those districts for which it has increased, there are often less than 10 or 20 buildings included in the dataset, indicating those numbers are likely not as reliable. On average, however, Brooklyn and Queens have improved the most with a median 2011 to 2015 decrease in EUI of 6.6% and 7.1%, whereas Manhattan and the Bronx have only improved by 3.9% and 2.5%, respectively.

WHAT ABOUT THE IMPACT OF LOCAL LAW 87?

From these maps we can see that on average most areas of NYC are decreasing their energy consumption, but we expect to see an even larger decrease for properties that have had to comply with Local Law 87 (requires a building undergo an ASHRAE level II energy audit and subsequent implementation of recommended retro-commissioning measures every 10 years).

Comparison of Properties with and without LL87 Submission

In the figure below we compare NYC buildings that had to comply with Local Law 87 in 2013 and 2014 (based on the block number) with those that did not. While the Local Law 87 buildings have around the same median EUI in the first four years (prior to and during the audit and retro-commissioning period), there appears to be a larger decrease in consumption for those buildings in 2015. Presumably, as data from additional years comes in, these buildings should continue to show improved performance.

ARE BENCHMARKED BUILDINGS BECOMING MORE EFFICIENT?

Based on this analysis of Local Law 84 benchmarking disclosure data from the last five years, energy consumption is decreasing in large multifamily buildings across the city at a rate of 1% per year (or 4% over five years). This is an encouraging trend! Our analysis is similar to the finding from NYC’s most recent Energy and Water Use Report, which reported a 5% decrease in energy consumption for multifamily buildings from 2010 to 2013. While the results differ slightly, it is based on data from different years, and as seen in this blog, the data cleaning process can have a large impact on the overall result, as including or excluding a certain year’s data can significantly change the composition of the dataset.

To summarize, the analysis here suggests that the worst buildings are improving the most, and buildings in Brooklyn and Queens are improving more quickly than those in Manhattan and the Bronx. Data for the most recent year also indicates that buildings complying with Local Law 87 have an even larger decrease in EUI than the rest of the large multifamily building population. Stay tuned as data from subsequent years could strengthen this trend. It’s important to stress that although NYC’s multifamily buildings have been decreasing their energy consumption since the implementation of Local Law 84, there could be many reasons for this besides merely the effect of benchmarking. These may include the effects of Local Law 87, energy prices, building code changes, the phasing out of fuel oil, incentive programs and the cost of rent. On the other hand, a recent paper published by the National Electrical Manufacturers Association (NEMA), suggests that a majority of NYC large multifamily building managers are changing their operating practices and making capital investments in energy efficiency as a result of energy benchmarking. A more detailed analysis could explore the effects of these different factors on the energy consumption trend.

Whatever the myriad reasons for the decrease in energy consumption, this is an ongoing process, which the city needs to continue monitoring each year. Building owners can play their part by tracking their buildings’ consumption on a monthly basis. We all have a role to play and we cannot be passive (unless it’s passive house!) about our actions. Stay tuned for a more detailed analysis from the City, the Urban Green Council and CUSP. If this post has sparked any ideas, interest or questions, please reach out!

1 It is worth noting that in the previous blog post exploring Local Law 84 data, we chose to use Site EUI, since that metric is more representative of how energy is being used at a building, and the goal of that exploration was to compare trends in building characteristics and energy consumption. In this analysis, however, we are using Source EUI, since weather normalized Site EUI is not available in all six years of disclosure data.

04 Oct

EnergyScoreCards Grading FAQ’s

Sarah Newman data, software 0 Comments

EnergyScoreCards grades enable users to target their efforts and resources towards the buildings in their portfolio with the greatest potential for savings. This is done by comparing buildings to their ‘peer’ buildings, assigning grades of A, B, C or D according to how their energy and water use stacks up to these peers. Our new algorithm strives to improve on our concept of a ‘peer’ building by more closely comparing buildings to those that have similar needs.

For more information about the new grading algorithm itself please click here.

Why is my building’s grade different?

The new grade is a more accurate picture of how your building is performing compared to peer buildings. You can think of this new grade as a building’s potential for savings. So a C or a D building likely has a lot of ‘low-hanging fruit’ measures that are faster and cheaper to implement, while an A or B building is performing well relative to peer buildings – it may still have opportunities to lower energy and water use, but they may take a bit more investment of time and money.

My building’s grades improved with the new algorithm, what does that mean?

Your building is likely of a type that tends to use more energy, so when it’s compared to other buildings like it, its relative energy use is not quite as substantial. This includes older, small buildings or buildings with lots of amenity space.

My building’s grades got worse with the new algorithm, what does that mean?

Your building likely is the type that tends to use less energy, so when compared to other similar buildings, we can see that there is more room for improvement. These include newer, large buildings without a lot of amenity space and all-tenant-paid garden style complexes.

What does a grade change mean for my site staff and how do I communicate this change to them?

If you’re already using EnergyScoreCards grades to motivate or inform site staff, this shouldn’t be too big of a change. Most grades either won’t change at all or by much. For buildings that do have a larger change in grades, it likely won’t be too much of a surprise to site staff, since they know your buildings best. The most important thing to communicate is that we’re continually updating our models to make the grades more useful, and this latest change is really aimed at making the grades as actionable as possible, so buildings that now have Cs or Ds should have a lot of potential for savings.

Why did the algorithm change?

We’re constantly updating our software and modeling algorithms to incorporate the most state-of-the-art methods possible. This latest change was also driven by a lot of feedback we’ve received from clients, both in terms of grades that didn’t make sense and wanting tighter regional comparison.

Why did only some of my building’s grades change?

Since this upgrade is really a refinement of our current model, many grades won’t change much or at all. It’s also possible that your overall energy grade might stay the same, but end use grades (heating, cooling, baseload electric and baseload fossil fuel) might change. This is because the end-use and overall energy grades are calculated separately, and could differ for many reasons, such as:

  • The distribution of energy use (and energy needs) between end-uses is often not equal.
  • There can be no energy used for a specific end-use, giving a property an end-use grade of ‘N/A’, but that means less energy overall that will factor into the overall grade.
  • You can’t tell if you’re on the border between two grades, so you might be just barely a ‘B’ for end-uses, but just barely an ‘A’ for overall energy.
  • Certain types of fuel are more efficient than others, so a property may have an efficient fossil fuel baseload score, but compared to properties that have those end-uses filled by electric, may not be as efficient overall.
  • Depending on the payment code, the metrics for end-uses may be measured against different building areas. For example, for TTOO buildings, the cooling index (and grade) is normalized by common area, whereas the overall energy index is normalized by total multi-family area.

How does this impact energy improvement recommendations I’ve already received from Bright Power (such as through an energy audit)?

EnergyScoreCards provides a high level analysis of building energy use. While EnergyScoreCards grades are an important input for our engineers, there are a lot of factors that go into producing the recommendations that go into an energy audit report, including equipment surveys, conversations with site staff, observations from a site visit, energy modeling, and our experience upgrading similar buildings. The energy improvement recommendations (or energy conservation measures) presented in an energy audit were custom created for your building using a lot more data than is available in EnergyScoreCards. EnergyScoreCards is a great place to keep track of these recommendations and to measure the actual energy performance improvements after you have implemented them.

How does this compare to Portfolio Manager’s Energy Star score calculation?

Portfolio Manager also uses a Machine Learning Regression algorithm to calculate their Energy Star score. However, we’re using a different type of algorithm that performs much better with different types of building data, and we’re ‘training’ our algorithm on a much larger database, making it both more accurate and better able to properly model many different types of buildings. We’re also only comparing buildings to those in their climate region, and we have a custom model for each of those regions. The EnergyScoreCards grading model also grades by energy end-uses (heating, cooling, baseload electric and baseload fossil fuel) in addition to total energy and water use, so you can see which parts of your energy consumption have the biggest potential for improvement. Finally, we’re able to grade buildings based on both the owner-paid portion of utility use or whole-building data, whereas whole-building data is required for an Energy Star score.

When is this rolling out?  

October 2016

Will historical grades change to the new algorithm?

Yes.

Will I be able to access old grades?

Not on the EnergyScoreCards website. Your EnergyScoreCards Energy Analyst can supply you with a spreadsheet showing your Most Recent Year grades before and after the change for reference.

03 Oct

The Science Behind the Grades

Sarah Newman benchmarking, data, energy management, software 2 Comments

scorecard

EnergyScoreCards, Bright Power’s premiere utility bill analytics software, is releasing an upgrade to its proprietary grading algorithm in October 2016. With this upgrade comes improved accuracy, refined regional analysis, and a focus on the potential for saving energy and water at every building. We’re excited to release this upgrade and give our clients an even deeper, more sophisticated understanding of energy performance past, present and projected. In this blog, we’ll take a deep dive into our new grading methodology and explore the nuances of multifamily building energy analysis.

Why do we grade buildings?

Making intelligent decisions about how to manage energy and water at the portfolio level requires distinguishing between the good, the bad, and the liability in terms of efficiency. If a property owner has two buildings on the same block with very different energy usage, does that mean that the one using less energy is performing closer to its potential? Is it even worth looking into energy saving measures in the ‘better’ performer? What if the buildings are very different ages or have different types of tenants? Or what if one building is in NYC and the other is in Los Angeles?

EnergyScoreCards grades help answer these types of questions and allow you to understand how efficiently your buildings are performing, and where best to focus your efforts and resources within a portfolio to maximize savings. We do this by comparing energy and water consumption metrics for your properties to those of similar buildings in our database of over 20,000 multifamily buildings.

As anyone who knows multifamily buildings can tell you, defining what “similar buildings” means is easier said than done. Multifamily buildings vary in terms of their location, size, age, construction, system types, amenities provided, the size and configuration of apartments, and many other physical and operational factors. On top of that, a variety of metering and payment structures means that in some cases we have access to whole building data, and in others only the portion of consumption paid for by owners. All of these factors and more must be taken into account in order to develop a meaningful and accurate grading methodology.

As our database grows, we are better able to understand the energy and water needs of all different kinds of multifamily buildings across the country. We also continually update our software to incorporate the most state-of-the-art tools available to provide value for our customers. As such, we have recently updated our EnergyScoreCards grading algorithm and it’s our most substantial change to the grading system yet. Our new model is able to better define the concept of a ‘similar’ building and uses new statistical tools to figure out which characteristics are the most important when it comes to grading. With the new model in effect, grades will be ‘fairer’ and will better indicate potential savings by being based on things that owners can actually change.

The EnergyScoreCards Dataset

Our database includes over 20,000 buildings with 800,000 units grouped into more than 6,000 properties, each with over 30 data fields such as total square feet, building age and resident demographics. By analyzing all of this data, we can figure out which building characteristics have the biggest impact on energy needs, and use these to calculate building grades.

To get an idea of the geographic span of our dataset, here’s a map of the U.S. with bubbles representing properties grouped by location.

This map shows the distribution of properties in the EnergyScoreCards database, with the size of the bubbles representing the number of properties in a given location.

Why this matters to you

This recent upgrade to EnergyScoreCards is part of Bright Power’s ongoing efforts to continually improve our data analysis and software to give building owners all of the information and guidance they need to better manage energy and water at their buildings. With improved peer comparison, our grades allow for:

  • More targeted audits
  • Better distribution of resources across a portfolio
  • Continued confidence in our ability to understand your building’s needs

Stay tuned for more improvements, we’ve got so much more on the way!

For more on how we’re better defining building peer groups, read on below:

buildings

Which building characteristics affect energy use?

One of the main features of EnergyScoreCards is the building energy grades, which allow property owners to understand how their properties compare to other buildings in terms of energy and water use. But determining a fair way to compare properties to similar buildings can be challenging. Our goal is to figure out what information is relevant to building energy use and how we can use this to group buildings with their peers.

There are many factors which contribute to a building’s energy use. Many of these are permanent factors, such as the age of the building or its size, and many are things that a building owner or manager can change, such as the specific heating, cooling and distribution equipment in the building. Our new algorithm does a better job of normalizing for the permanent features of a building, so that the grade is only based on the things that an owner can change – the fixable factors. So now you can know if the huge electric bill at your 30-story building is because it’s not performing efficiently or if it’s in line with other tall buildings with lots of elevators.

Our new algorithm also breaks up our dataset into more geographic regions that have similar climates and energy use patterns. You can see the new regions in the map below.

This map shows the different geographic regions used by our grading algorithm.

But how can we tell which building characteristics are important for determining how much energy a building should use? Fortunately, our new grading model can help answer this question, and ranks building characteristics on how much they impact a building’s energy and water use.

Once we understand what affects energy use, we can look at what types of buildings use more or less energy based on the most predictive characteristics. The graphs below show the Energy Use Index (EUI – kBTU/sqft/yr) vs different building characteristics, for all NYC buildings with owner-paid heat and hot-water. No discernable trends are apparent for any of these characteristics on their own…

These scatter plots show the relationship between EUI and average apartment size, total area and year built. None of these building characteristics alone appear to have a strong trend with EUI.

…but when we look at the EUI compared to multiple characteristics at once we can find ‘pockets’ of parameter space that stand out. In the graph below, each circle represents a group of buildings binned by average apartment size and the age of the building. The size of the circle represents the number of properties in the bin and the color represents the median EUI of the bin. From this, we can see that old buildings with small average apartment size have the most intense energy use. While buildings with small average apartment size built around the 1960s also have relatively high EUIs, these constitute a relatively small number of properties (as shown by the circle size), and so it’s difficult to tell if this is a strong trend. On average though, energy use becomes more efficient with increasing apartment size for buildings of any age, which makes sense as you have fewer residents occupying the same space.

This graph shows buildings binned by average apartment size and the age of the building, each bin shown as a circle. The size of the circle represents the number of properties in the bin and the color represents the median EUI of the bin.

Similarly, the graph below shows how EUI varies with total building area and the average apartment size, and we see a trend of small buildings with small average apartment size using energy the least efficiently.

This graph shows buildings binned by average apartment size and the total multifamily area, each bin shown as a circle. The size of the circle represents the number of properties in the bin and the color represents the median EUI of the bin.

These graphs allow us to identify types of buildings that require more or less energy per square foot, even when operating efficiently. We can explore this trend further by looking at the distribution of EUIs for two ‘types’ of buildings:

  • Old, small buildings with small apartments
  • New, large buildings with large apartments

While there is some overlap in the distributions, there is definitely a tendency for old, small buildings with small apartments to use more energy per sqft. In our model, we want to grade these buildings fairly, so they’ll only be getting C’s and D’s if their use is inefficient compared to buildings like them, and not compared to all buildings.

The two lines show the smoothed distributions of EUI values for different subsets of owner-paid heat and hot water buildings in NYC from our database. The purple line shows the distribution for old (before 1950), small buildings (less than 100,000 sqft) with small apartments (less than 1000 sqft), and the green line shows the distribution for new, large buildings with large apartments. To view an interactive version of this graph, click here.

How do we use this information to grade your buildings?

In order to fairly compare buildings to their peers, we calculate each building’s predicted EUI based on its permanent characteristics using our entire database of buildings. This predicted EUI can be thought of as its peer building’s use. To calculate this predicted EUI, we use a Machine Learning algorithm called a Random Forest Regressor. This algorithm is effective at fitting both numerical characteristics (such as building square feet) and categorical characteristics (such as the type of fuel used for end-uses), and it’s also not as susceptible to overfitting as other types of models (something data scientists always worry about). A Random Forest Regressor is constructed of many decision trees, each of which determines the energy use of a building by traversing through the tree based on all of its characteristics. The final predicted EUI is then calculated from the average predicted value from all of the 200 trees in a given model. Since our grading algorithm uses around 60 models total, that means we’re calculating the predicted EUI for buildings in our database using 12,000 different Decision Trees!

A portion of an example decision tree is shown below.


Example Decision Tree

To determine the predicted EUI for a given property, traverse the tree from left to right. At each node, take the top or bottom branch, depending on whether the statement in the node is true or false. For example, for a property with the following characteristics:

  • Average apartment size = 1300 sqft
  • Total size = 800,000 sqft
  • Number of Units = 250
  • Year built = 1950

You would traverse the tree, by answering: TRUE, TRUE, TRUE, FALSE


We then calculate the grades based on how much a building’s actual EUI compares to the predicted EUI, so the grades represent how much a building’s energy use differs from a typical peer building based on changeable factors.

With our improved peer comparison, you can think of your building’s energy grade as its potential for savings. A ‘D’ really just means you could be saving a lot of money! You wouldn’t expect your aging apartment building in Minnesota to ever have the same energy consumption per area as a new garden development in California, but with the right focused effort, it can still get an ‘A’.

To download the pdf version of this article, click here.