What challenges arise when deploying a novel technology at increasing
                          scale? This case study details our experience developing and deploying
                          technologies to monitor power outages and voltage fluctuations at high
                          temporal and geographic frequency. After a small initial pilot, our
                          deployment grew over time and eventually exceeded 450 sensors and 3500
                          mobile app downloads with households and firms across Accra, Ghana. Our
                          first lesson is that ad hoc solutions to deployment challenges may not
                          scale, as larger scales bring unique challenges requiring unique and
                          progressively more complex solutions. Second, challenges that arise with
                          scale span distinct domains – not only technological but also cultural,
                          organizational, and operational. Finally, we stress the importance of
                          adaptability of operational structure, and of frequently updating
                          operational strategy based on new learnings.
                        

Content has been formatted as TeX source. Click to format for web.

What challenges arise when deploying a novel technology at increasing scale? This case study details our experience developing and deploying technologies to monitor power outages and voltage fluctuations at high temporal and geographic frequency. After a small initial pilot, our deployment grew over time and eventually exceeded 450 sensors and 3500 mobile app downloads with households and firms across Accra, Ghana. Our first lesson is that ad hoc solutions to deployment challenges may not scale, as larger scales bring unique challenges requiring unique and progressively more complex solutions. Second, challenges that arise with scale span distinct domains – not only technological but also cultural, organizational, and operational. Finally, we stress the importance of adaptability of operational structure, and of frequently updating operational strategy based on new learnings.

Content has been formatted for web. Click to view raw TeX

                          Complicated systems are complicated to monitor. The electric grid is one
                          of the most complicated systems, and subsequently goes under-monitored
                          in many regions around the world that cannot easily afford expensive
                          meters. However, the electric grid is also critical for sustaining a
                          high quality of life, and requires better monitoring than is often
                          available to ensure consistent service is provided. Past work has shown
                          that images taken by satellites during the night, capturing nighttime
                          illumination (“nightlights”), could provide a proxy measurement of grid
                          performance for minimal cost. We build upon earlier work by identifying
                          the pixel z-score – a statistical measurement of a pixel's illumination
                          relative to its history – as a key method for detecting electricity
                          outages from the often-noisy nightlights dataset. We then train and
                          validate our approach against observations from a network of
                          on-the-ground power outage sensors in our observation area of Accra,
                          Ghana, a dataset representing the largest collection of
                          utility-independent electricity reliability measurements on the African
                          continent. Using multiple machine learning techniques for estimating
                          potential outages from nightlight images, we obtain high performance for
                          predicting outages in Accra at scales as small as a single pixel (0.2
                          km) and with training datasets as small as three months of
                          illumination/sensor data. We further validate our methodology beyond the
                          spatio-temporal coverage of the on-the-ground sensor deployment against
                          a human-labeled dataset of outages by neighborhood throughout Accra.
                          Delving deeper into the applications and limitations of available
                          datasets and our work, we conclude by highlighting questions about the
                          generality of our method vital to understanding its potential for
                          low-cost worldwide measurements of grid reliability.
                        

Content has been formatted as TeX source. Click to format for web.

Complicated systems are complicated to monitor. The electric grid is one of the most complicated systems, and subsequently goes under-monitored in many regions around the world that cannot easily afford expensive meters. However, the electric grid is also critical for sustaining a high quality of life, and requires better monitoring than is often available to ensure consistent service is provided. Past work has shown that images taken by satellites during the night, capturing nighttime illumination (“nightlights”), could provide a proxy measurement of grid performance for minimal cost. We build upon earlier work by identifying the pixel z-score – a statistical measurement of a pixel's illumination relative to its history – as a key method for detecting electricity outages from the often-noisy nightlights dataset. We then train and validate our approach against observations from a network of on-the-ground power outage sensors in our observation area of Accra, Ghana, a dataset representing the largest collection of utility-independent electricity reliability measurements on the African continent. Using multiple machine learning techniques for estimating potential outages from nightlight images, we obtain high performance for predicting outages in Accra at scales as small as a single pixel (0.2 km) and with training datasets as small as three months of illumination/sensor data. We further validate our methodology beyond the spatio-temporal coverage of the on-the-ground sensor deployment against a human-labeled dataset of outages by neighborhood throughout Accra. Delving deeper into the applications and limitations of available datasets and our work, we conclude by highlighting questions about the generality of our method vital to understanding its potential for low-cost worldwide measurements of grid reliability.

Content has been formatted for web. Click to view raw TeX

                          In much of the world, electricity grids are not instrumented at the
                          customer level, limiting insights into the power quality experienced by
                          utility customers. Moreover, to understand grid performance, regulators
                          and investors must depend on utilities to self-report reliability data.
                          To address these challenges, we introduce PowerWatch, an agile
                          methodology to directly measure customer experience and aggregated grid
                          performance without relying on the utility for deployment or management.
                          PowerWatch employs a system of distributed sensors coupled with
                          cloud-based analytics. We evaluate the PowerWatch methodology by
                          deploying 462 sensors in homes and businesses in Accra, Ghana for over a
                          year, yielding the largest open-source data set on electricity
                          reliability at the customer-level in the region. We describe the
                          architecture, design, and performance of PowerWatch, as well as the data
                          that are collected, explaining how we determine the accuracy and
                          coverage of our methodology without ground truth. Finally, we report on
                          grid performance issues, finding nearly twice as many outages as the
                          utility observed, suggesting a need for better grid performance
                          monitoring.
                        

Content has been formatted as TeX source. Click to format for web.

In much of the world, electricity grids are not instrumented at the customer level, limiting insights into the power quality experienced by utility customers. Moreover, to understand grid performance, regulators and investors must depend on utilities to self-report reliability data. To address these challenges, we introduce PowerWatch, an agile methodology to directly measure customer experience and aggregated grid performance without relying on the utility for deployment or management. PowerWatch employs a system of distributed sensors coupled with cloud-based analytics. We evaluate the PowerWatch methodology by deploying 462 sensors in homes and businesses in Accra, Ghana for over a year, yielding the largest open-source data set on electricity reliability at the customer-level in the region. We describe the architecture, design, and performance of PowerWatch, as well as the data that are collected, explaining how we determine the accuracy and coverage of our methodology without ground truth. Finally, we report on grid performance issues, finding nearly twice as many outages as the utility observed, suggesting a need for better grid performance monitoring.

Content has been formatted for web. Click to view raw TeX

                          The vision of sensor systems that collect critical and previously ungathered
                          information about the world is often only realized when sensors, students,
                          and subjects move outside the academic laboratory. However, deployments at
                          even the smallest scales introduce complexities and risks that can be
                          difficult for a research team to anticipate. Over the past year, our
                          interdisciplinary team of engineers and economists has been designing,
                          deploying, and operating a large sensor network in Accra, Ghana that
                          measures power outages and quality at households and firms. This network
                          consists of 457 custom sensors, over 3,000 mobile app instances,
                          thousands of participant surveys, and custom user incentive and
                          deployment management systems. In part, this deployment supports an
                          evaluation of the impacts of investments in the grid on reliability and
                          the subsequent effects of improvements in reliability on
                          socioeconomic well-being. We report our experiences as we move from
                          performing small pilot deployments to our current scale, attempting to
                          identify the pain points at each stage of the deployment. Finally, we
                          extract high-level observations and lessons learned from our
                          deployment activities, which we wish we had originally known when
                          forecasting budgets, human resources, and project timelines. These
                          insights will be critical as we look toward scaling our deployment to the
                          en-tire city of Accra and beyond, and we hope that they will encourage and
                          support other researchers looking to measure highly granular information
                          about our world's critical systems.
                        

Content has been formatted as TeX source. Click to format for web.

The vision of sensor systems that collect critical and previously ungathered information about the world is often only realized when sensors, students, and subjects move outside the academic laboratory. However, deployments at even the smallest scales introduce complexities and risks that can be difficult for a research team to anticipate. Over the past year, our interdisciplinary team of engineers and economists has been designing, deploying, and operating a large sensor network in Accra, Ghana that measures power outages and quality at households and firms. This network consists of 457 custom sensors, over 3,000 mobile app instances, thousands of participant surveys, and custom user incentive and deployment management systems. In part, this deployment supports an evaluation of the impacts of investments in the grid on reliability and the subsequent effects of improvements in reliability on socioeconomic well-being. We report our experiences as we move from performing small pilot deployments to our current scale, attempting to identify the pain points at each stage of the deployment. Finally, we extract high-level observations and lessons learned from our deployment activities, which we wish we had originally known when forecasting budgets, human resources, and project timelines. These insights will be critical as we look toward scaling our deployment to the en-tire city of Accra and beyond, and we hope that they will encourage and support other researchers looking to measure highly granular information about our world's critical systems.

Content has been formatted for web. Click to view raw TeX

                          Incentives are a key facet of human studies research, yet the
                          state-of-the-art often designs and implements incentive systems in an
                          ad-hoc, on-demand manner. We introduce the first vocabulary for formally
                          describing incentive systems and develop a software infrastructure that
                          enables UI-based graphical generation of complex, auditable, reliable,
                          and reproducible incentive systems. We call this infrastructure the Open
                          INcentive Kit (OINK). A review of recent literature from several
                          communities finds that of the one hundred and twenty-one publications
                          that incorporate incentives, only thirty-one describe their incentive
                          system in detail, and all of these could be implemented using OINK. We
                          evaluate OINK in practice by using it for an active energy monitoring
                          deployment in Ghana and find that OINK successfully facilitates
                          thousands of individual incentive payments. Finally, we describe our
                          efforts to generalize OINK for different research communities,
                          specifically focusing on architectural decisions around extensibility to
                          support unanticipated use cases. OINK is free and open-source software.
                        

Content has been formatted as TeX source. Click to format for web.

Incentives are a key facet of human studies research, yet the state-of-the-art often designs and implements incentive systems in an ad-hoc, on-demand manner. We introduce the first vocabulary for formally describing incentive systems and develop a software infrastructure that enables UI-based graphical generation of complex, auditable, reliable, and reproducible incentive systems. We call this infrastructure the Open INcentive Kit (OINK). A review of recent literature from several communities finds that of the one hundred and twenty-one publications that incorporate incentives, only thirty-one describe their incentive system in detail, and all of these could be implemented using OINK. We evaluate OINK in practice by using it for an active energy monitoring deployment in Ghana and find that OINK successfully facilitates thousands of individual incentive payments. Finally, we describe our efforts to generalize OINK for different research communities, specifically focusing on architectural decisions around extensibility to support unanticipated use cases. OINK is free and open-source software.

Content has been formatted for web. Click to view raw TeX

                          Network connectivity is often one of the most challenging aspects of
                          deploying sensors. In many countries, cellular networks provide the
                          most reliable, highest bandwidth, and greatest coverage option for
                          internet access. While this makes smartphones a seemingly ideal
                          platform to serve as a gateway between sensors and the cloud, we
                          find that a device designed for multi-tenant operation and frequent
                          human interaction becomes unreliable when tasked to continuously run
                          a single application with no human interaction, a seemingly
                          counter-intuitive result. Further, we find that economy phones
                          cannot physically withstand continuous operation, resulting in a
                          surprisingly high rate of permanent device failures in the field. If
                          these observations hold more broadly, they would make mobile phones
                          poorly suited to a range of sensing applications for which they have
                          been rumored to hold great promise.
                        

Content has been formatted as TeX source. Click to format for web.

Network connectivity is often one of the most challenging aspects of deploying sensors. In many countries, cellular networks provide the most reliable, highest bandwidth, and greatest coverage option for internet access. While this makes smartphones a seemingly ideal platform to serve as a gateway between sensors and the cloud, we find that a device designed for multi-tenant operation and frequent human interaction becomes unreliable when tasked to continuously run a single application with no human interaction, a seemingly counter-intuitive result. Further, we find that economy phones cannot physically withstand continuous operation, resulting in a surprisingly high rate of permanent device failures in the field. If these observations hold more broadly, they would make mobile phones poorly suited to a range of sensing applications for which they have been rumored to hold great promise.

Content has been formatted for web. Click to view raw TeX