Case Study: University of California Irvine overhauls BAS, adds FDD

Welcome to our Case Study series, where we dive into case studies of real-life, large-scale deployments of smart building technologies, supported by the Nexus Marketplace.

I emphasize “real life” because this isn’t a marketing fluff story. We're here to share real lessons from leaders who have done the work to integrate smart building technology into their operations. I also emphasize “large scale” because we're not here to talk about pilot projects. We're here to talk about deeper commitments to changing how buildings are operated.

---

Case Study Data:

• Technology Categories Mentioned: Service Layer - Commissioning Agent, Service Layer - Network Manager, Application Layer - Supervisory Control, Application Layer - FDD, Network Layer, Device Layer - HVAC Control
• Key Stakeholders: Joseph Fleshman, Interim Director of Energy Engineering at the University of California, Irvine (UCI). Jim Meacham, Principal at Altura Associates
• Vendors: Altura Associates for master systems integrator (MSI), commissioning (Cx) agent, and fault detection diagnostics (FDD) services; Tridium Niagara as the building automation front end for supervisory control and visibility of building automation; SkySpark (also by Altura) as a cloud-based analytics and FDD overlay.
• Number of Buildings: 23 buildings complete, equating to 25% of the UCI campus
• Project Dates: 2016 to Present

Case Study Outline:

• Introduction
• Background
• Technical Overview
• Challenges & Lessons Learned
• Conclusion

---

University of California, Irvine (UCI) is a 36,000-student, 224-degree public university in Irvine, California. UCI has partnered with Altura Associates, an engineering advisor and trusted master systems integrator (MSI). Since 2016, these two entities have been collaborating on a campus-wide project to bring the campus facilities technology to the forefront of the industry. 

We sat down with Joseph Fleshman of UCI and Jim Meacham of Altura Associates to break down the story of transitioning this university to the modern era of building automation, what obstacles this created, and how UCI continues to benefit from this impactful project.

Background

Since 2007, UCI has been investing heavily in energy savings projects. However, their building technologies were holding them back. 

They needed more consistency from building to building and project to project in the building automation stack. Lack of consistency made it difficult for the facilities team to deploy energy-saving control sequences at scale. Keeping up with the status and performance of equipment took too much time, with point-by-point manual commissioning needing to be more efficient and scalable. The UCI team realized that modernizing the BAS architecture and monitoring performance with FDD software were required to meet UCI’s energy-saving goals. 

Fleshman explained, “Benefits to the university are twofold. The first is energy efficiency. Through having a more standardized infrastructure and bringing data into SkySpark for fault detection and diagnostics, we’ve been able to continue the energy savings improvements that UCI has really been known for. ”

Additionally, UCI is considered an industry leader in smart laboratories, which require critical climate control for accurate and efficient experiments, further emphasizing the importance of a reliable and modern building automation system.

In 2017, UCI and Altura Associates began the implementation of FDD and Supervisory Control on campus to have standardized and rich building data.

FDD software allows for advanced trending and automated commissioning. These tools improve building efficiency by finding issues and offering corrective actions. The insights on system performance that FDD can provide help the university prioritize future building efficiency projects and help ensure that crucial research facilities are in good shape, keeping research productive. 

Finally, FDD helps improve the user experience at UCI, as Fleshman puts it, “Through fault detection and diagnostics, we are able to better identify issues with the buildings before users do. So for hot and cold calls, we’re able to diagnose them quicker, or even identify them before the users have the opportunity to call and complain”.

Niagara, a device-agnostic building automation platform, gives UCI a consistent look and feel for their supervisory control without relying on any proprietary protocols or silo’d systems. This consistent platform streamlines the jobs of building operations staff. Transitioning to Niagara has improved UCI’s ability to create competitive solicitation options for devices and implementation services. The competitive solicitation not only improves pricing, helping UCI’s overall budget, but it also shortens project timelines since the university is no longer reliant on the schedule of a single vendor.

UCI and Altura are about 25% of the way into scaling this project across the entire campus. UCI is saving approximately $750,000 in annual energy costs through this implementation. The growing building data available in SkySpark has allowed significant automated testing, easing the incorporation of advanced control sequences like temperature and pressure resets and uncovering failing equipment.

As the expanding project has gained momentum, UCI has been able to settle on agreed-upon and proven standards and specifications for new construction and renovation projects. This standardization has resulted in clear indications of improved Operational Technology (OT) network stability and simplified future project work for the remaining 80% of the campus.

Technical Overview

Technically, this project can be split into three categories, all overlapping throughout the chronological implementation.

#1 - Overhauling the Building Automation System

The old building automation approach, used by UCI and many other buyers, is fragmented building-by-building. Multiple automation vendors would install proprietary solutions consisting of building-level servers talking to supervisory controllers, which talk to field-level controls.

This method creates control sequences, point naming, and graphics that are often proprietary, not easily updated or accessible, and inconsistent. Additionally, the facilities team responsible for managing these buildings didn’t have input into what solutions are implemented in new projects.

Enter the new era of BAS: moving supervisory control up the stack via the establishment of standardized graphics, point naming, and control sequencing. UCI and Altura established Niagara as the consistent supervisory platform across all buildings.

Note: If you’re new to this style of BAS transition, we highly recommend you check out some resources in the Nexus Library, like a previous blog featuring Altura “The BAS Architecture of the Future”, and our holistic Buyers Guide to HVAC Controls.

Fleshman emphasizes the importance of this non-proprietary and consistent approach to BAS and how it allowed for competitive bid solicitation, product/vendor flexibility, and easier remote access to the building automation system for the UCI team. 

Meacham and the Altura team have significant experience bringing customers to this new era of BAS. Meacham compared this modern BAS approach to how enterprise customers treat an Enterprise Resource Planning (ERP) system: 

“I think what we're realizing, particularly with campus clients or large corporate institutional clients, is just like any other enterprise system, they need to treat building automation controls like a truly enterprise system. You wouldn't have, for example, an SAP system per building, right?”

The obstacle that most clients have with this type of system overhaul is that budgets for institutions owning multiple buildings can often be split building by building, which makes it challenging to think of your building automation system as an enterprise system.

With the university leadership bought-in on the concept, Fleshman and his team started taking small bites out of the BAS overhaul by using the maintenance budget to update any proprietary field level controllers to be BACnet compatible. Fleshman doubled down on the importance of BACnet compatibility: many building owners will find legacy devices speaking proprietary protocols speckled throughout their facilities. To get consistency, UCI needed to get all systems capable of communicating together, which is typically on a BACnet/IP network.

However, as more and more BACnet-enabled devices were added to the network, communication traffic skyrocketed, and devices began falling offline, which led to the second technical category of the project.

#2 - Overhauling OT Networking

The original OT network topology of UCI went hand in hand with its original building automation topology: littered with silos. Specifically, UCI had 4 flat, local networks, each representing the footprint of a specific proprietary BAS vendor. These networks had a mix of BACnet and non-BACnet devices.

The new standard became for all devices to speak BACnet/IP. The topology would consist of multiple campus-wide servers to help support scalability and a single graphical user interface. But things got loud as new BACnet devices came online.

“It’s basically like you had a bunch of people in an arena all shouting at each other and none of the devices can hear themselves, so they would drop offline”
—Joseph Fleshman

As new BACnet devices depleted network reliability, developing new networking standards became imminent. Altura and UCI spent over a year working with the UCI IT department to define parameters around how the OT network would operate in a way that satisfied security and other organizational requirements. 

For example, all building devices would share a common first and second octet within their IP address, and the 3rd octet would represent each unique building. BACnet instance ID assignment was pre-defined for each device with the support of the IT department. And perhaps most critically, each building was isolated onto its own network with only one outlet through the server. This means that all devices can communicate within a building, but only a server can communicate information outside of the building.

Adopting these new OT rules to all projects on campus was not a simple feat. Specification updates were required for all maintenance and capital projects across many divisions within the specifications, including divisions 23, 25, 26, 27, and 11.

Continuing to support the development of this new OT network topology, new projects were specified to no longer include supervisory level controllers, like JACEs. Meacham mentioned how, initially, JACEs were approved devices supporting the move of supervisory control up the stack. But UCI and Altura quickly recognized that these devices were unnecessary given the improved network architecture, and these devices locked UCI to Niagara as a long-term BAS vendor, since JACEs work specifically with the Niagara platform. Even though the university is happy with its network platform, the goal of vendor agnosticism was the deciding factor, and JACEs were removed from specifications so that UCI was capable of moving vendors if a better solution than Niagara came in the future.

The last major aspect of the OT Network overhaul was the connection of the SkySpark VPN. The SkySpark platform, which relies on the large amounts of data it receives from the campus, is hosted in the cloud, outside of the greater UCI network. This required major security discussions and a procedure that satisfied the IT department.

As the connection to SkySpark became active, the platform became capable of providing value to UCI, which brings us to the 3rd technical category of the project.

#3 Deploying FDD and Automated Commissioning

When UCI had localized and silo’d BAS servers, the trending of data was difficult. Trends were customized and set up individually. However, as building automation data moved up the stack, Altura could support with specific recommendations of what to trend within the SkySpark platform, which alleviated the manpower UCI needed to support data collection.

SkySpark’s internal analytics began to unleash the power of FDD. SkySpark data improved the CMMS information that the UCI automation team was able to provide to HVAC techs in the field, adding more efficiency to maintenance workflows. Additionally, reliable trending data has allowed UCI to mitigate finger-pointing when, for example, expensive labs have issues. 

Fleshman shared a specific example of an expensive piece of lab equipment that lost a run because temperature and humidity changed too fast within the room in which it operated. Within SkySpark, Fleshman’s team was able to determine the root cause for corrective action immediately. Meacham offered a useful comparison of this feature of SkySpark to the flight data recorder information available in airplanes.

Beyond FDD, SkySpark opens UCI to the world of automated commissioning. For example, automated commissioning allows UCI to send BACnet commands to large swaths of devices during unoccupied hours to identify equipment anomalies. Fleshman walked through a specific example: “UCI was able to identify which zones weren’t behaving the way they were supposed to… for 100% of the building… it’s a much more technologically advanced way of doing things than in the past, where we couldn’t hit every single zone of the building”. 

The commissioning style of the early 2000s often included a single commissioning agent holding a single temperature probe to check a single discharge temperature. SkySpark automated commissioning migrates these tasks to the digital world and allows them to be repeated on every piece of equipment, not just a selected sampling.

UCI even has additional benefits from the SkySpark platform outside of typical building FDD. The university continues to connect smart energy meters to the platform, which allows UCI to bill on-campus customers for their energy usage. Valuable central utility plant (CUP) trending has also moved to the SkySpark cloud, providing valuable insights into campus performance to UCI leadership and utility companies.

Challenges & Lessons Learned

As anyone would expect on a project of this size, it didn’t come without unexpected challenges. Fleshman and Meacham shared some valuable insights on the hindsight they’ve gained since starting the project.

Lesson #1 - UCI should’ve started by defining and adopting a comprehensive OT network topology

Addressing network instability was not the project's goal; however, it was discovered by implementing SkySpark and Niagara. Fleshman and Meacham indicated that starting with the IT team earlier, setting a standard, and managing the network earlier would’ve alleviated some of the early network reliability issues.

Meacham explains this as an “all of the above approach,” meaning defining network specifications in a way that allows you to make devices and buildings available on the FDD platform during minor and major capital projects and renovations… all the above.

Fleshman further explained the preparation a facility manager should have for the investment required when getting to a BACnet network requires new hardware. Initially, facilities personnel might need to be made aware of what equipment is BACnet or can be switched to BACnet versus what will need to be ripped and replaced. One approach Fleshman suggested for other building owners is starting your new network with any equipment that’s already natively BACnet while you line up funding to replace hardware that isn’t BACnet compatible. 

Moreover, building owners should budget for BACnet requirements during any large capital projects —don’t let new equipment miss the new standard. Fleshman had a specific example, where, in 2018, 4 different 60,000-square-foot renovations occurred, including major HVAC upgrades to laboratories. Fleshman and his team were able to get ahead of the project and build a SkySpark connection into the budget, which massively accelerated the amount of equipment available for FDD and automated testing.

Lesson #2 - Network security complexities associated with creating a business-to-business SkySpark VPN

Similar to lesson #1, Fleshman indicated that additional early and frequent communication with the IT departments could have saved more time, specifically about the crucial connections outside of the campus network to the cloud. Moving supervisory control up the stack and implementing FDD will almost undoubtedly require cloud connections outside of the local network. This means that building owners need to anticipate this early and get approval and contextual understanding from those responsible for IT firewalls and security.

Lesson #3 - Managing multiple vendors and enforcing new standards

Before this project, each of UCI’s BAS vendors had their own style when it came to automation projects. They were not standardized on the look of the graphics, the point naming convention, or the programming. Treating BAS like an enterprise-level system relies on vendors who understand and agree with the goal. Meacham indicated that building owners need to appreciate the amount of training that vendors and facilities staff will need to get on board with the new methodology. In the case of UCI, one new construction building slipped through the cracks and was completed using old specifications, resulting in devices not prepared to be part of the larger network.

Meacham noted that if vendors understand the reasons behind the new standards, they typically want to be supportive and abide. They want what’s best for the customer. 

The effort of enforcing new standards is a story of knocking down organizational silos. Fleshman and Meacham reflected on the number of people, internal and external to UCI, from many different organizations who had to be brought into the vision to create a successful project.

Conclusion

Nexus Labs has continued to educate on the importance of breaking down silos, both technologically and culturally, to move the industry forward. The success at UCI is a prime example of putting that mantra into practice. 

On a technical level, forward-thinking project implementation can lead to device layer vendor agnosticism, saving institutions money and time on projects while enhancing flexibility.

Successful energy savings, decarbonization, and facilities workflow simplification are rarely just about the smart devices and software we all know and love. Equally important, if not more important, are the spheres of influence surrounding those products to make them successful. 

At UCI, this includes building an OT network topology that allows thousands of devices to communicate efficiently and without failure. It includes an IT department that understands the needs of your buildings and supports the development of communication standards. It includes an ecosystem of vendors and service providers that understand and support the long-term stack goals and are set up to succeed when applying these goals to retrofit and new construction projects, both large and small.