Category: benchmarking

01 May

Exploring Citywide Energy Usage Part II

Sarah Newman benchmarking, data, energy, LL84 0 Comments

HOW HAS CITYWIDE MULTIFAMILY ENERGY CONSUMPTION CHANGED?

In a recent blog post, we explored how energy consumption varies geographically across New York City using the publicly disclosed Local Law 84 data from 2015. In this follow-up, we look at how energy use has changed in the years that the benchmarking law has been in place. Using publicly disclosed data for the last four years and data provided by the NYC Office of Sustainability for the first two, we are able to assess how the law continues to shape New York City buildings for the better.

Data Cleaning Methodology

In this analysis, we use weather-normalized Source EUI1 as calculated by Energy Star’s Portfolio Manager software, which will just be referred to as “EUI” in the remainder of the blog. Given that the first year’s data (2010) reduces the overall dataset size by over half and is the most suspect in terms of data quality, we opt to instead include only the last five years of energy disclosure data. In addition, we require that buildings have a valid EUI for each of the last five years and remove buildings that vary in their reported square footage by more than 10% between any two years or in their EUI by more than 60%, either of which would indicate that a data entry error is likely. We also remove the top and bottom 1% of EUIs from all years pooled together. This leaves a total of 2,282 multifamily buildings in our dataset.

Change in Median Source EUI

After cleaning the data, we analyze the change in the median EUI over the last five years (see figure below), fitting it with an ordinary least squares regression. The best fit model implies a decrease in the median EUI of 1.3 kBTU/sqft/yr each year (or 1%/year), with a total decrease in EUI of 5.2 kBTU/sqft/yr over the full 5 years. This linear model has an adjusted r² value of 0.75, indicating a robust fit.

This change in the median EUI can be thought of as how much the average building population is changing its consumption overall. We can also look at the median change in EUI from 2011 to 2015 (6.0 kBTU/sqft/yr or 4.6% over the five-year span), which is more representative of how a typical building has changed over those five years.

Table 1 shows the median EUI for each year. It is important to note that these values vary somewhat from those published in NYC’s Benchmarking reports as the former are derived only from buildings with valid data in all five years.

Table 1. Median Source EUI from 2011-2015

    Year     Median Source
EUI (kBTU/sqft/yr)
2011 129.9
2012 126.3
2013 125.8
2014 124.2
2015 124.4

Looking Across All Six Years

If we instead include all six years of data, the decrease in median EUI is a bit larger (1.7 kBTU/sqft/yr/yr), but as noted before, this reduces the overall dataset size by more than half to only 808 buildings. The histogram below shows the distribution of 2015 EUIs for three different data subsets, as well as the median for each subset (vertical dashed lines):

  • Buildings with data in years 2012-2015
  • Buildings with data in years 2011-2015 (the dataset used throughout this blog)
  • Buildings with data in all 6 years

While there is not a large difference in the total number or distribution of EUIs for the first two groups, the subset including only buildings with data in all six years has a much higher median EUI for the overlapping five years (127.8 vs 124.4 for 2015), meaning that buildings that complied in the first year have on average higher EUIs than those that didn’t.

Comparing the Change for the Best and Worst Buildings

We can also see the shift in the EUI over the last five years by looking at the change in the normalized EUI distribution with time. The figure below displays the normalized EUI distribution (from a Kernel-density estimate) for each year, showing that the EUI distribution becomes narrower with a lower median EUI over time.

In order to confirm that this trend is not just year-to-year noise in building consumption, we want to see that the worst buildings are improving significantly and the best buildings are not getting worse. We can see this is the case in Table 2, which shows the median 2011 and 2015 EUIs of buildings in the 1st, 2nd, 3rd and 4th quartiles of buildings from the 2011 data.

Table 2. Change in Median Source EUI for 2011 Quartiles

2011  Quartiles 

2011 Median Source EUI (kBTU/sqft/yr) 2015 Median Source EUI (kBTU/sqft/yr) 2011 to 2015 EUI % Difference

1st

98.3 100.6

2%

2nd

120.5 115.6

-4%

3rd

140.7

132.4

-6%

4th 172.7 155.8

-10%

While the buildings in the best quartile have increased their usage by 2%, the remaining 3/4 of buildings have improved, with the worst quartile of buildings decreasing their usage by 10% – suggesting that overall, buildings in NYC are improving.

This reduction in energy consumption can also be evaluated with a paired t-test, comparing the distributions of EUIs in 2011 and 2015, which returns a p-value of 6×10−28, indicating that the distributions of EUIs in those two years are statistically distinct.

Excluding the Impact of Hurricane Sandy

Thus far in this analysis, we have not compensated for the effects of Hurricane Sandy which cut out power to many areas in NYC for weeks in 2012. In the figure below, we remove areas with prolonged outages (Red Hook, The Rockaways and Lower Manhattan) from the dataset and compare the EUI trend between that set and our main dataset.

The ‘Sandy-corrected’ group, which omits heavily affected areas, shows the same trend of decreasing EUI over time, albeit with an even steeper slope (1.45 kBTU/sqft/yr vs 1.3 kBTU/sqft/yr). The median EUI for each year is also slightly lower for the Sandy-corrected group (see Table 3), likely because buildings in lower Manhattan have relatively high EUIs (see interactive map in the following section).

Table 3. Median Source EUI for Sandy-corrected Data

  Year   

Median Source EUI
(kBTU/sqft/yr)
Median Source EUI
Sandy-corrected (kBTU/sqft/yr)

2011

129.9 128.7

2012

126.3

125.2

2013 125.8

124.5

2014 124.2

122.7

2015

124.4

122.7

ARE ALL PARTS OF NYC IMPROVING?

We can also explore how this yearly change in EUI varies geographically. Are all boroughs and neighborhoods in NYC improving at the same pace? Are some still getting worse?

2011-2015 Median Source EUI by Community District

The maps below show the median EUI for each NYC community district for each year from 2011 to 2015 as well as the absolute and % difference from 2011 to 2015. You can click on the tabs at the top of the map to switch years, and hover over a community district to see the median EUI and number of buildings.

From these maps we can see that energy consumption has decreased in most regions, and in those districts for which it has increased, there are often less than 10 or 20 buildings included in the dataset, indicating those numbers are likely not as reliable. On average, however, Brooklyn and Queens have improved the most with a median 2011 to 2015 decrease in EUI of 6.6% and 7.1%, whereas Manhattan and the Bronx have only improved by 3.9% and 2.5%, respectively.

WHAT ABOUT THE IMPACT OF LOCAL LAW 87?

From these maps we can see that on average most areas of NYC are decreasing their energy consumption, but we expect to see an even larger decrease for properties that have had to comply with Local Law 87 (requires a building undergo an ASHRAE level II energy audit and subsequent implementation of recommended retro-commissioning measures every 10 years).

Comparison of Properties with and without LL87 Submission

In the figure below we compare NYC buildings that had to comply with Local Law 87 in 2013 and 2014 (based on the block number) with those that did not. While the Local Law 87 buildings have around the same median EUI in the first four years (prior to and during the audit and retro-commissioning period), there appears to be a larger decrease in consumption for those buildings in 2015. Presumably, as data from additional years comes in, these buildings should continue to show improved performance.

ARE BENCHMARKED BUILDINGS BECOMING MORE EFFICIENT?

Based on this analysis of Local Law 84 benchmarking disclosure data from the last five years, energy consumption is decreasing in large multifamily buildings across the city at a rate of 1% per year (or 4% over five years). This is an encouraging trend! Our analysis is similar to the finding from NYC’s most recent Energy and Water Use Report, which reported a 5% decrease in energy consumption for multifamily buildings from 2010 to 2013. While the results differ slightly, it is based on data from different years, and as seen in this blog, the data cleaning process can have a large impact on the overall result, as including or excluding a certain year’s data can significantly change the composition of the dataset.

To summarize, the analysis here suggests that the worst buildings are improving the most, and buildings in Brooklyn and Queens are improving more quickly than those in Manhattan and the Bronx. Data for the most recent year also indicates that buildings complying with Local Law 87 have an even larger decrease in EUI than the rest of the large multifamily building population. Stay tuned as data from subsequent years could strengthen this trend. It’s important to stress that although NYC’s multifamily buildings have been decreasing their energy consumption since the implementation of Local Law 84, there could be many reasons for this besides merely the effect of benchmarking. These may include the effects of Local Law 87, energy prices, building code changes, the phasing out of fuel oil, incentive programs and the cost of rent. On the other hand, a recent paper published by the National Electrical Manufacturers Association (NEMA), suggests that a majority of NYC large multifamily building managers are changing their operating practices and making capital investments in energy efficiency as a result of energy benchmarking. A more detailed analysis could explore the effects of these different factors on the energy consumption trend.

Whatever the myriad reasons for the decrease in energy consumption, this is an ongoing process, which the city needs to continue monitoring each year. Building owners can play their part by tracking their buildings’ consumption on a monthly basis. We all have a role to play and we cannot be passive (unless it’s passive house!) about our actions. Stay tuned for a more detailed analysis from the City, the Urban Green Council and CUSP. If this post has sparked any ideas, interest or questions, please reach out!

1 It is worth noting that in the previous blog post exploring Local Law 84 data, we chose to use Site EUI, since that metric is more representative of how energy is being used at a building, and the goal of that exploration was to compare trends in building characteristics and energy consumption. In this analysis, however, we are using Source EUI, since weather normalized Site EUI is not available in all six years of disclosure data.

03 Oct

The Science Behind the Grades

Sarah Newman benchmarking, data, energy management, software 2 Comments

scorecard

EnergyScoreCards, Bright Power’s premiere utility bill analytics software, is releasing an upgrade to its proprietary grading algorithm in October 2016. With this upgrade comes improved accuracy, refined regional analysis, and a focus on the potential for saving energy and water at every building. We’re excited to release this upgrade and give our clients an even deeper, more sophisticated understanding of energy performance past, present and projected. In this blog, we’ll take a deep dive into our new grading methodology and explore the nuances of multifamily building energy analysis.

Why do we grade buildings?

Making intelligent decisions about how to manage energy and water at the portfolio level requires distinguishing between the good, the bad, and the liability in terms of efficiency. If a property owner has two buildings on the same block with very different energy usage, does that mean that the one using less energy is performing closer to its potential? Is it even worth looking into energy saving measures in the ‘better’ performer? What if the buildings are very different ages or have different types of tenants? Or what if one building is in NYC and the other is in Los Angeles?

EnergyScoreCards grades help answer these types of questions and allow you to understand how efficiently your buildings are performing, and where best to focus your efforts and resources within a portfolio to maximize savings. We do this by comparing energy and water consumption metrics for your properties to those of similar buildings in our database of over 20,000 multifamily buildings.

As anyone who knows multifamily buildings can tell you, defining what “similar buildings” means is easier said than done. Multifamily buildings vary in terms of their location, size, age, construction, system types, amenities provided, the size and configuration of apartments, and many other physical and operational factors. On top of that, a variety of metering and payment structures means that in some cases we have access to whole building data, and in others only the portion of consumption paid for by owners. All of these factors and more must be taken into account in order to develop a meaningful and accurate grading methodology.

As our database grows, we are better able to understand the energy and water needs of all different kinds of multifamily buildings across the country. We also continually update our software to incorporate the most state-of-the-art tools available to provide value for our customers. As such, we have recently updated our EnergyScoreCards grading algorithm and it’s our most substantial change to the grading system yet. Our new model is able to better define the concept of a ‘similar’ building and uses new statistical tools to figure out which characteristics are the most important when it comes to grading. With the new model in effect, grades will be ‘fairer’ and will better indicate potential savings by being based on things that owners can actually change.

The EnergyScoreCards Dataset

Our database includes over 20,000 buildings with 800,000 units grouped into more than 6,000 properties, each with over 30 data fields such as total square feet, building age and resident demographics. By analyzing all of this data, we can figure out which building characteristics have the biggest impact on energy needs, and use these to calculate building grades.

To get an idea of the geographic span of our dataset, here’s a map of the U.S. with bubbles representing properties grouped by location.

This map shows the distribution of properties in the EnergyScoreCards database, with the size of the bubbles representing the number of properties in a given location.

Why this matters to you

This recent upgrade to EnergyScoreCards is part of Bright Power’s ongoing efforts to continually improve our data analysis and software to give building owners all of the information and guidance they need to better manage energy and water at their buildings. With improved peer comparison, our grades allow for:

  • More targeted audits
  • Better distribution of resources across a portfolio
  • Continued confidence in our ability to understand your building’s needs

Stay tuned for more improvements, we’ve got so much more on the way!

For more on how we’re better defining building peer groups, read on below:

buildings

Which building characteristics affect energy use?

One of the main features of EnergyScoreCards is the building energy grades, which allow property owners to understand how their properties compare to other buildings in terms of energy and water use. But determining a fair way to compare properties to similar buildings can be challenging. Our goal is to figure out what information is relevant to building energy use and how we can use this to group buildings with their peers.

There are many factors which contribute to a building’s energy use. Many of these are permanent factors, such as the age of the building or its size, and many are things that a building owner or manager can change, such as the specific heating, cooling and distribution equipment in the building. Our new algorithm does a better job of normalizing for the permanent features of a building, so that the grade is only based on the things that an owner can change – the fixable factors. So now you can know if the huge electric bill at your 30-story building is because it’s not performing efficiently or if it’s in line with other tall buildings with lots of elevators.

Our new algorithm also breaks up our dataset into more geographic regions that have similar climates and energy use patterns. You can see the new regions in the map below.

This map shows the different geographic regions used by our grading algorithm.

But how can we tell which building characteristics are important for determining how much energy a building should use? Fortunately, our new grading model can help answer this question, and ranks building characteristics on how much they impact a building’s energy and water use.

Once we understand what affects energy use, we can look at what types of buildings use more or less energy based on the most predictive characteristics. The graphs below show the Energy Use Index (EUI – kBTU/sqft/yr) vs different building characteristics, for all NYC buildings with owner-paid heat and hot-water. No discernable trends are apparent for any of these characteristics on their own…

These scatter plots show the relationship between EUI and average apartment size, total area and year built. None of these building characteristics alone appear to have a strong trend with EUI.

…but when we look at the EUI compared to multiple characteristics at once we can find ‘pockets’ of parameter space that stand out. In the graph below, each circle represents a group of buildings binned by average apartment size and the age of the building. The size of the circle represents the number of properties in the bin and the color represents the median EUI of the bin. From this, we can see that old buildings with small average apartment size have the most intense energy use. While buildings with small average apartment size built around the 1960s also have relatively high EUIs, these constitute a relatively small number of properties (as shown by the circle size), and so it’s difficult to tell if this is a strong trend. On average though, energy use becomes more efficient with increasing apartment size for buildings of any age, which makes sense as you have fewer residents occupying the same space.

This graph shows buildings binned by average apartment size and the age of the building, each bin shown as a circle. The size of the circle represents the number of properties in the bin and the color represents the median EUI of the bin.

Similarly, the graph below shows how EUI varies with total building area and the average apartment size, and we see a trend of small buildings with small average apartment size using energy the least efficiently.

This graph shows buildings binned by average apartment size and the total multifamily area, each bin shown as a circle. The size of the circle represents the number of properties in the bin and the color represents the median EUI of the bin.

These graphs allow us to identify types of buildings that require more or less energy per square foot, even when operating efficiently. We can explore this trend further by looking at the distribution of EUIs for two ‘types’ of buildings:

  • Old, small buildings with small apartments
  • New, large buildings with large apartments

While there is some overlap in the distributions, there is definitely a tendency for old, small buildings with small apartments to use more energy per sqft. In our model, we want to grade these buildings fairly, so they’ll only be getting C’s and D’s if their use is inefficient compared to buildings like them, and not compared to all buildings.

The two lines show the smoothed distributions of EUI values for different subsets of owner-paid heat and hot water buildings in NYC from our database. The purple line shows the distribution for old (before 1950), small buildings (less than 100,000 sqft) with small apartments (less than 1000 sqft), and the green line shows the distribution for new, large buildings with large apartments. To view an interactive version of this graph, click here.

How do we use this information to grade your buildings?

In order to fairly compare buildings to their peers, we calculate each building’s predicted EUI based on its permanent characteristics using our entire database of buildings. This predicted EUI can be thought of as its peer building’s use. To calculate this predicted EUI, we use a Machine Learning algorithm called a Random Forest Regressor. This algorithm is effective at fitting both numerical characteristics (such as building square feet) and categorical characteristics (such as the type of fuel used for end-uses), and it’s also not as susceptible to overfitting as other types of models (something data scientists always worry about). A Random Forest Regressor is constructed of many decision trees, each of which determines the energy use of a building by traversing through the tree based on all of its characteristics. The final predicted EUI is then calculated from the average predicted value from all of the 200 trees in a given model. Since our grading algorithm uses around 60 models total, that means we’re calculating the predicted EUI for buildings in our database using 12,000 different Decision Trees!

A portion of an example decision tree is shown below.


Example Decision Tree

To determine the predicted EUI for a given property, traverse the tree from left to right. At each node, take the top or bottom branch, depending on whether the statement in the node is true or false. For example, for a property with the following characteristics:

  • Average apartment size = 1300 sqft
  • Total size = 800,000 sqft
  • Number of Units = 250
  • Year built = 1950

You would traverse the tree, by answering: TRUE, TRUE, TRUE, FALSE


We then calculate the grades based on how much a building’s actual EUI compares to the predicted EUI, so the grades represent how much a building’s energy use differs from a typical peer building based on changeable factors.

With our improved peer comparison, you can think of your building’s energy grade as its potential for savings. A ‘D’ really just means you could be saving a lot of money! You wouldn’t expect your aging apartment building in Minnesota to ever have the same energy consumption per area as a new garden development in California, but with the right focused effort, it can still get an ‘A’.

To download the pdf version of this article, click here.

26 May

Why Did I Get an LL84 Benchmarking Violation?!

Conor Laver benchmarking, LL84 0 Comments

IMG_2754

Energy benchmarking and disclosure laws are getting real.  To date, NYC’s Local Law 84 (LL84), like most benchmarking laws, has required buildings to submit data on their annual energy usage without checking to see if the data was complete or sensible. The bar was set low to ease building owners into this reporting requirement. But now that the law has been around for a few years, the City is shifting from requiring mere participation to requiring real, useful data.  This year, LL84 benchmarking submissions are being held to a higher data quality standard than in previous years and violations are being issued to those who do not meet that standard.  And we can only assume that other cities with energy benchmarking and disclosure laws will follow suit.  Why is this happening, and why is it happening now?

What We’ve Seen

Since NYC’s benchmarking law was first introduced, there has been a proliferation of low-cost operators who promise to just “make the LL84 problem go away”, by submitting the bare minimum necessary to avoid a fine. I won’t name any names but we have seen some shocking examples of negligence from these operators, who submit totally false data to the City. When you see a submission that claims a building has only used 1000 gallons of oil this year for all of it’s energy, and no electricity at all, it’s obvious that something is up. We see cases like this all the time, and we expect to see more as the violations go out and we’re hired to correct the submissions. While these operators are temporarily assuaging a pain point for building owners, the result of their service is detrimental not only to their clients, but also the City.

Why This Matters

Let’s not forget that LL84 was born of a genuine need to gather energy data which would inform the City’s plans for improvement. Given the volume of low-quality submissions, a large portion of the submitted data is useless, making it all the more difficult to identify the greatest areas of need.

This data is crucial to building a better, more sustainable city. Retrofitting Affordability, co-authored by The Building Energy Exchange and Bright Power, is a great example of what can be accomplished with accurate data. Through data analysis, we were able to identify which buildings and which energy efficiency retrofit measures have the greatest potential for carbon reduction, an important step in executing the City’s climate action plan, One City: Built to Last. Through this plan, the City is committed to improving the building stock. Without accurate data, everyone is working blind.

This data is also incredibly valuable to real estate owners, for much the same reason. Accurate data is the cornerstone of any energy management strategy, from planning to execution and verification.  There’s a common saying about analyzing bad data: Garbage In, Garbage Out.

What To Do If You Received a Violation

The LL84 benchmarking violations will now result from either failure to submit data or data quality errors such as:

  1. Missing or zero in the ‘Property Floor Area ‘field (square feet);
  2. Missing or zero in the ‘Number of Buildings’ field;
  3. Missing or zero in the ‘Site EUI (kBTU/sq. ft.)’ field (Site Energy Use Intensity);
  4.  Missing or zero in the ‘Source EUI (kBTU/sq. ft.)’ field (Source Energy Use Intensity);
  5. Data in the ‘Metered Areas (Energy)’ field is missing or does not account for the total energy consumption
  6.  Missing or incorrectly formatted BBL number

If you’ve received a benchmarking violation, in NYC or elsewhere, and are wondering what do next, don’t be angry at the City; this data is crucial to making your building better and the City better. Instead come talk to us, about not only getting benchmarking done right, but also turning it from an annoying annual expense, into a valuable insight. You will pay less, the city will burn less fuel and the only people losing out will be the paper-pushing middlemen, who enjoyed getting rich without providing value.