This list contains select peer-reviewed and internal publications.
Some were written before nLine was started, and represent our
foundational ideas.
Scientific integrity is a core value and we will continue to publish
our learnings. Check back for internal studies, documents produced
for customers, and opinion pieces, all currently under development.
2022
-
Measuring Grid Reliability in Ghana
Noah Klugman, Joshua Adkins, Susanna Berkouwer, Kwame Abrokwah, Matthew Podolsky, Pat Pannuto, Catherine Wolfram, Jay Taneja, and Prabal Dutta
Introduction to Development Engineering: A Framework with Applications from the Field - [chapter]
@Inbook{Klugman2023, author="Klugman, Noah and Adkins, Joshua and Berkouwer, Susanna and Abrokwah, Kwame and Podolsky, Matthew and Pannuto, Pat and Wolfram, Catherine and Taneja, Jay and Dutta, Prabal", editor="Madon, Temina and Gadgil, Ashok J. and Anderson, Richard and Casaburi, Lorenzo and Lee, Kenneth and Rezaee, Arman", title="Measuring Grid Reliability in Ghana", bookTitle="Introduction to Development Engineering: A Framework with Applications from the Field", year="2023", publisher="Springer International Publishing", address="Cham", pages="129--159", abstract="What challenges arise when deploying a novel technology at increasing scale? This case study details our experience developing and deploying technologies to monitor power outages and voltage fluctuations at high temporal and geographic frequency. After a small initial pilot, our deployment grew over time and eventually exceeded 450 sensors and 3500 mobile app downloads with households and firms across Accra, Ghana. Our first lesson is that ad hoc solutions to deployment challenges may not scale, as larger scales bring unique challenges requiring unique and progressively more complex solutions. Second, challenges that arise with scale span distinct domains -- not only technological but also cultural, organizational, and operational. Finally, we stress the importance of adaptability of operational structure, and of frequently updating operational strategy based on new learnings.", isbn="978-3-030-86065-3", doi="10.1007/978-3-030-86065-3_6", url="https://doi.org/10.1007/978-3-030-86065-3_6" }
What challenges arise when deploying a novel technology at increasing scale? This case study details our experience developing and deploying technologies to monitor power outages and voltage fluctuations at high temporal and geographic frequency. After a small initial pilot, our deployment grew over time and eventually exceeded 450 sensors and 3500 mobile app downloads with households and firms across Accra, Ghana. Our first lesson is that ad hoc solutions to deployment challenges may not scale, as larger scales bring unique challenges requiring unique and progressively more complex solutions. Second, challenges that arise with scale span distinct domains – not only technological but also cultural, organizational, and operational. Finally, we stress the importance of adaptability of operational structure, and of frequently updating operational strategy based on new learnings.
Content has been formatted as TeX source. Click to format for web.
What challenges arise when deploying a novel technology at increasing scale? This case study details our experience developing and deploying technologies to monitor power outages and voltage fluctuations at high temporal and geographic frequency. After a small initial pilot, our deployment grew over time and eventually exceeded 450 sensors and 3500 mobile app downloads with households and firms across Accra, Ghana. Our first lesson is that ad hoc solutions to deployment challenges may not scale, as larger scales bring unique challenges requiring unique and progressively more complex solutions. Second, challenges that arise with scale span distinct domains – not only technological but also cultural, organizational, and operational. Finally, we stress the importance of adaptability of operational structure, and of frequently updating operational strategy based on new learnings.
-
The electricity scene from above: Exploring power grid inconsistencies using satellite data in Accra, Ghana
Zeal Shaha, Noah Klugman, Gabriel Cadamuro, Feng-Chi Hsu, Christopher D. Elvidge, Jay Taneja
Applied Energy - [paper]
@article{SHAH2022119237, title = {The electricity scene from above: Exploring power grid inconsistencies using satellite data in Accra, Ghana}, journal = {Applied Energy}, volume = {319}, pages = {119237}, year = {2022}, issn = {0306-2619}, doi = {https://doi.org/10.1016/j.apenergy.2022.119237}, url = {https://www.sciencedirect.com/science/article/pii/S0306261922005980}, author = {Zeal Shah and Noah Klugman and Gabriel Cadamuro and Feng-Chi Hsu and Christopher D. Elvidge and Jay Taneja} }
Complicated systems are complicated to monitor. The electric grid is one of the most complicated systems, and subsequently goes under-monitored in many regions around the world that cannot easily afford expensive meters. However, the electric grid is also critical for sustaining a high quality of life, and requires better monitoring than is often available to ensure consistent service is provided. Past work has shown that images taken by satellites during the night, capturing nighttime illumination (“nightlights”), could provide a proxy measurement of grid performance for minimal cost. We build upon earlier work by identifying the pixel z-score – a statistical measurement of a pixel's illumination relative to its history – as a key method for detecting electricity outages from the often-noisy nightlights dataset. We then train and validate our approach against observations from a network of on-the-ground power outage sensors in our observation area of Accra, Ghana, a dataset representing the largest collection of utility-independent electricity reliability measurements on the African continent. Using multiple machine learning techniques for estimating potential outages from nightlight images, we obtain high performance for predicting outages in Accra at scales as small as a single pixel (0.2 km) and with training datasets as small as three months of illumination/sensor data. We further validate our methodology beyond the spatio-temporal coverage of the on-the-ground sensor deployment against a human-labeled dataset of outages by neighborhood throughout Accra. Delving deeper into the applications and limitations of available datasets and our work, we conclude by highlighting questions about the generality of our method vital to understanding its potential for low-cost worldwide measurements of grid reliability.
Content has been formatted as TeX source. Click to format for web.
Complicated systems are complicated to monitor. The electric grid is one of the most complicated systems, and subsequently goes under-monitored in many regions around the world that cannot easily afford expensive meters. However, the electric grid is also critical for sustaining a high quality of life, and requires better monitoring than is often available to ensure consistent service is provided. Past work has shown that images taken by satellites during the night, capturing nighttime illumination (“nightlights”), could provide a proxy measurement of grid performance for minimal cost. We build upon earlier work by identifying the pixel z-score – a statistical measurement of a pixel's illumination relative to its history – as a key method for detecting electricity outages from the often-noisy nightlights dataset. We then train and validate our approach against observations from a network of on-the-ground power outage sensors in our observation area of Accra, Ghana, a dataset representing the largest collection of utility-independent electricity reliability measurements on the African continent. Using multiple machine learning techniques for estimating potential outages from nightlight images, we obtain high performance for predicting outages in Accra at scales as small as a single pixel (0.2 km) and with training datasets as small as three months of illumination/sensor data. We further validate our methodology beyond the spatio-temporal coverage of the on-the-ground sensor deployment against a human-labeled dataset of outages by neighborhood throughout Accra. Delving deeper into the applications and limitations of available datasets and our work, we conclude by highlighting questions about the generality of our method vital to understanding its potential for low-cost worldwide measurements of grid reliability.
2021
-
Watching the Grid: Utility-Independent Measurements of Electricity Reliability in Accra, Ghana
Noah Klugman, Joshua Adkins, Emily Paszkiewicz, Molly G. Hickman, Matthew Podolsky, Jay Taneja, and Prabal Dutta
International Conference on Information Processing in Sensor Networks (co-located with CPS-IoT Week 2021) (IPSN’21) - [paper] [slides (as pdf)] [conference]
@inbook{klugman21watching, author = {Klugman, Noah and Adkins, Joshua and Paszkiewicz, Emily and Hickman, Molly G. and Podolsky, Matthew and Taneja, Jay and Dutta, Prabal}, title = {Watching the Grid: Utility-Independent Measurements of Electricity Reliability in Accra, Ghana}, year = {2021}, isbn = {9781450380980}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3412382.3458276}, booktitle = {Proceedings of the 20th International Conference on Information Processing in Sensor Networks (Co-Located with CPS-IoT Week 2021)}, pages = {341–356}, numpages = {16} }
In much of the world, electricity grids are not instrumented at the customer level, limiting insights into the power quality experienced by utility customers. Moreover, to understand grid performance, regulators and investors must depend on utilities to self-report reliability data. To address these challenges, we introduce PowerWatch, an agile methodology to directly measure customer experience and aggregated grid performance without relying on the utility for deployment or management. PowerWatch employs a system of distributed sensors coupled with cloud-based analytics. We evaluate the PowerWatch methodology by deploying 462 sensors in homes and businesses in Accra, Ghana for over a year, yielding the largest open-source data set on electricity reliability at the customer-level in the region. We describe the architecture, design, and performance of PowerWatch, as well as the data that are collected, explaining how we determine the accuracy and coverage of our methodology without ground truth. Finally, we report on grid performance issues, finding nearly twice as many outages as the utility observed, suggesting a need for better grid performance monitoring.
Content has been formatted as TeX source. Click to format for web.
In much of the world, electricity grids are not instrumented at the customer level, limiting insights into the power quality experienced by utility customers. Moreover, to understand grid performance, regulators and investors must depend on utilities to self-report reliability data. To address these challenges, we introduce PowerWatch, an agile methodology to directly measure customer experience and aggregated grid performance without relying on the utility for deployment or management. PowerWatch employs a system of distributed sensors coupled with cloud-based analytics. We evaluate the PowerWatch methodology by deploying 462 sensors in homes and businesses in Accra, Ghana for over a year, yielding the largest open-source data set on electricity reliability at the customer-level in the region. We describe the architecture, design, and performance of PowerWatch, as well as the data that are collected, explaining how we determine the accuracy and coverage of our methodology without ground truth. Finally, we report on grid performance issues, finding nearly twice as many outages as the utility observed, suggesting a need for better grid performance monitoring.
2019
-
Hardware, Apps, and Surveys at Scale: Insights from Measuring Grid Reliability in Accra, Ghana
Noah Klugman, Joshua Adkins, Susanna Berkouwer, Kwame Abrokwah, Ivan Bobashev, Pat Pannuto, Matthew Podolsky, Aldo Susenot, Revati Thatte, Catherine Wolfram, Jay Taneja, and Prabal Dutta
ACM SIGCAS Conference on Computing and Sustainable Societies (COMPASS’19) - [paper] [slides (as pdf)] [conference]
@inproceedings{klugman19scale, booktitle = {ACM SIGCAS Conference on Computing and Sustainable Societies}, title = {Hardware, Apps, and Surveys at Scale: Insights from Measuring Grid Reliability in Accra, Ghana}, series = {COMPASS'19}, year = {2019}, month = {July}, location = {Accra, Ghana}, conference-url = {https://acmcompass.org/}, author = {Klugman, Noah and Adkins, Joshua and Berkouwer, Susanna and Abrokwah, Kwame and Bobashev, Ivan and Pannuto, Pat and Podolsky, Matthew and Susenot, Aldo and Thatte, Revati and Wolfram, Catherine and Taneja, Jay and Dutta, Prabal}, }
The vision of sensor systems that collect critical and previously ungathered information about the world is often only realized when sensors, students, and subjects move outside the academic laboratory. However, deployments at even the smallest scales introduce complexities and risks that can be difficult for a research team to anticipate. Over the past year, our interdisciplinary team of engineers and economists has been designing, deploying, and operating a large sensor network in Accra, Ghana that measures power outages and quality at households and firms. This network consists of 457 custom sensors, over 3,000 mobile app instances, thousands of participant surveys, and custom user incentive and deployment management systems. In part, this deployment supports an evaluation of the impacts of investments in the grid on reliability and the subsequent effects of improvements in reliability on socioeconomic well-being. We report our experiences as we move from performing small pilot deployments to our current scale, attempting to identify the pain points at each stage of the deployment. Finally, we extract high-level observations and lessons learned from our deployment activities, which we wish we had originally known when forecasting budgets, human resources, and project timelines. These insights will be critical as we look toward scaling our deployment to the en-tire city of Accra and beyond, and we hope that they will encourage and support other researchers looking to measure highly granular information about our world's critical systems.
Content has been formatted as TeX source. Click to format for web.
The vision of sensor systems that collect critical and previously ungathered information about the world is often only realized when sensors, students, and subjects move outside the academic laboratory. However, deployments at even the smallest scales introduce complexities and risks that can be difficult for a research team to anticipate. Over the past year, our interdisciplinary team of engineers and economists has been designing, deploying, and operating a large sensor network in Accra, Ghana that measures power outages and quality at households and firms. This network consists of 457 custom sensors, over 3,000 mobile app instances, thousands of participant surveys, and custom user incentive and deployment management systems. In part, this deployment supports an evaluation of the impacts of investments in the grid on reliability and the subsequent effects of improvements in reliability on socioeconomic well-being. We report our experiences as we move from performing small pilot deployments to our current scale, attempting to identify the pain points at each stage of the deployment. Finally, we extract high-level observations and lessons learned from our deployment activities, which we wish we had originally known when forecasting budgets, human resources, and project timelines. These insights will be critical as we look toward scaling our deployment to the en-tire city of Accra and beyond, and we hope that they will encourage and support other researchers looking to measure highly granular information about our world's critical systems.
-
The Open INcentive Kit (OINK): Standardizing the Generation, Comparison, and Deployment of Incentive Systems
Noah Klugman, Santiago Correa, Pat Pannuto, Matthew Podolsky, Jay Taneja, and Prabal Dutta
The Tenth International Conference on Information and Communication Technologies and Development (ICTD’19) - [paper] [slides (as pdf)] [conference]
@inproceedings{klugman19oink, booktitle = {The Tenth International Conference on Information and Communication Technologies and Development}, title = {The {Open} {INcentive} {Kit} {(OINK)}: Standardizing the Generation, Comparison, and Deployment of Incentive Systems}, series = {ICTD'19}, year = {2019}, month = {1}, location = {Ahmedabad, India}, conference-url = {https://www.ictdx.org/}, author = {Klugman, Noah and Correa, Santiago and Pannuto, Pat and Podolsky, Matthew and Taneja, Jay and Dutta, Prabal}, }
Incentives are a key facet of human studies research, yet the state-of-the-art often designs and implements incentive systems in an ad-hoc, on-demand manner. We introduce the first vocabulary for formally describing incentive systems and develop a software infrastructure that enables UI-based graphical generation of complex, auditable, reliable, and reproducible incentive systems. We call this infrastructure the Open INcentive Kit (OINK). A review of recent literature from several communities finds that of the one hundred and twenty-one publications that incorporate incentives, only thirty-one describe their incentive system in detail, and all of these could be implemented using OINK. We evaluate OINK in practice by using it for an active energy monitoring deployment in Ghana and find that OINK successfully facilitates thousands of individual incentive payments. Finally, we describe our efforts to generalize OINK for different research communities, specifically focusing on architectural decisions around extensibility to support unanticipated use cases. OINK is free and open-source software.
Content has been formatted as TeX source. Click to format for web.
Incentives are a key facet of human studies research, yet the state-of-the-art often designs and implements incentive systems in an ad-hoc, on-demand manner. We introduce the first vocabulary for formally describing incentive systems and develop a software infrastructure that enables UI-based graphical generation of complex, auditable, reliable, and reproducible incentive systems. We call this infrastructure the Open INcentive Kit (OINK). A review of recent literature from several communities finds that of the one hundred and twenty-one publications that incorporate incentives, only thirty-one describe their incentive system in detail, and all of these could be implemented using OINK. We evaluate OINK in practice by using it for an active energy monitoring deployment in Ghana and find that OINK successfully facilitates thousands of individual incentive payments. Finally, we describe our efforts to generalize OINK for different research communities, specifically focusing on architectural decisions around extensibility to support unanticipated use cases. OINK is free and open-source software.
2018
-
Experience: Android Resists Liberation from Its Primary Use Case
Noah Klugman, Veronica Jacome, Meghan Clark, Matthew Podolsky, Pat Pannuto, Neal Jackson, Aley Soud Nassor, Catherine Wolfram, Duncan Callaway, Jay Taneja, and Prabal Dutta
The 24th Annual International Conference on Mobile Computing and Networking (MobiCom’18) - [paper] [slides (as pdf)] [conference]
@inproceedings{klugman18liberation, booktitle = {The 24th Annual International Conference on Mobile Computing and Networking}, title = {Experience: Android Resists Liberation from Its Primary Use Case}, series = {MobiCom'18}, year = {2018}, month = {10}, location = {New Delhi, India}, conference-url = {http://www.sigmobile.org/mobicom/2018/}, author = {Klugman, Noah and Jacome, Veronica and Clark, Meghan and Podolsky, Matthew and Pannuto, Pat and Jackson, Neal and Nassor, Aley Soud and Wolfram, Catherine and Callaway, Duncan and Taneja, Jay and Dutta, Prabal}, }
Network connectivity is often one of the most challenging aspects of deploying sensors. In many countries, cellular networks provide the most reliable, highest bandwidth, and greatest coverage option for internet access. While this makes smartphones a seemingly ideal platform to serve as a gateway between sensors and the cloud, we find that a device designed for multi-tenant operation and frequent human interaction becomes unreliable when tasked to continuously run a single application with no human interaction, a seemingly counter-intuitive result. Further, we find that economy phones cannot physically withstand continuous operation, resulting in a surprisingly high rate of permanent device failures in the field. If these observations hold more broadly, they would make mobile phones poorly suited to a range of sensing applications for which they have been rumored to hold great promise.
Content has been formatted as TeX source. Click to format for web.
Network connectivity is often one of the most challenging aspects of deploying sensors. In many countries, cellular networks provide the most reliable, highest bandwidth, and greatest coverage option for internet access. While this makes smartphones a seemingly ideal platform to serve as a gateway between sensors and the cloud, we find that a device designed for multi-tenant operation and frequent human interaction becomes unreliable when tasked to continuously run a single application with no human interaction, a seemingly counter-intuitive result. Further, we find that economy phones cannot physically withstand continuous operation, resulting in a surprisingly high rate of permanent device failures in the field. If these observations hold more broadly, they would make mobile phones poorly suited to a range of sensing applications for which they have been rumored to hold great promise.