New.

Geoffrey Challen
1 parent f092c4d9
Showing 5 changed files with 78 additions and 62 deletions
abstract.tex
conclusion.tex
include/start.tex
introduction.tex
results.tex
 \begin{abstract}
  
-While great strides have been made in measuring energy consumption through
-accurate energy models, measures of energy consumption alone are not
-sufficient to enable effective energy management on battery-constrained
-mobile devices. What is urgently needed is a way to put energy consumption in
-context by measuring the value delivered by mobile apps. An accurate value
-measure would enable cross-app comparison, app improvement, energy virus
-detection, and effective runtime energy allocation and prioritization. Given
-that gains in modeling will be lost without a measure of value, we believe
-that this is the most important open problem in energy management. Our paper
-motivates the problem, describes requirements for a candidate value measure,
-discusses possible inputs to such a metric, and presents result from a
-preliminary (unsuccessful) attempt to formulate a value measure.
+While great strides have been made in measuring energy consumption, these
+measures alone are not sufficient to enable effective energy management on
+battery-constrained mobile devices. What is urgently needed is a way to put
+energy consumption into context by measuring the \textit{value} delivered by
+mobile apps. While difficult to compute, an accurate value measure would
+enable cross-app comparison, app improvement, energy virus detection, and
+effective runtime energy allocation and prioritization. Our paper motivates
+the problem, describes requirements for a value measure, discusses and
+evaluates several possible inputs to such a measure, and presents results
+from a preliminary (unsuccessful) attempt to formulate one.
  
 \end{abstract}
 \section{Conclusions}
 \label{sec-conclusion}
  
-% \section*{Acknowledgments}
+To conclude, we have argued that our inability to estimate app value is a
+critical weakness that is threatening our successes at accurately estimating
+and attributing energy consumption. We have motivated the need for a value
+measure by describing the multiple ways in which it would aid in the
+management of energy and other resources on battery-powered smartphones.
+Using an energy consumption dataset collected on \PhoneLab{} we have explored
+separately several potential inputs to a value measure and determined how
+they weight energy consumption. And finally, we have presented results from a
+failed effort to formulate an effective value measure. While this first
+attempt was unsuccessful, we hope to engage the mobile systems community in
+this effort so that more sophisticated and successful value measures can be
+developed.
@@ -26,8 +26,8 @@
  
 \usepackage[all]{hypcap}
  
-\setlist[itemize]{leftmargin=*,partopsep=5pt}
-\setlist[enumerate]{leftmargin=*,partopsep=5pt}
+\setlist[itemize]{leftmargin=*,partopsep=0pt,topsep=5pt,itemsep=1pt}
+\setlist[enumerate]{leftmargin=*,partopsep=0pt,topsep=5pt,itemsep=1pt}
  
 \input{.xxxnote}
 \input{.draft}
 \section{Introduction}
  
-Measuring app energy consumption\footnote{To avoid confusion between device
-usage and energy usage, we use \textit{consumption} exclusively when
-referring to energy usage and \textit{usage} exclusively when referring to
-user interaction with the device or apps.} on mobile devices is close to
-being a solved problem, due to the great strides made in both generating and
-validating energy models that can deliver accurate runtime energy consumption
-estimates~\cite{mansdi, vedge-nsdi13} and in accurately attributing energy
-consumption---even for asynchronous and shared
-resources~\cite{cinder-eurosys11,osdi08-quanto}. 
-Previous work on component-based power modelling~\cite{dong2011, zhang2010, 
-jung2012} has mapped energy consumption to system-components like cpu, wifi chip, screen etc. 
-On the other hand, efforts like Eprof~\cite{pathak2011,pathak2012}, AppScope~\cite{yoon} traces system calls and monitors kernel activities to answer how much energy is consumed in an application level.
-Accurate energy models bring
-us one step close to the goal of effective energy management on smartphones
-and other battery-constrained mobile devices, while also providing developers
-with useful feedback as they build their mobile apps.
-
-But accurate energy measurement alone is not enough to enable these goals.
-Even perfectly-accurate measurements of energy consumption are insufficient
-to answer critical questions about app energy consumption faced by users and
-developers, including:
+Measuring app energy consumption\footnote{\small To avoid confusion between
+app and energy usage, we use \textit{consumption} exclusively when referring
+to energy usage and \textit{usage} exclusively when referring to user
+interaction with apps.} on mobile devices is close to being a solved problem,
+due to the great strides made in both generating and validating energy models
+that can deliver accurate runtime energy consumption
+estimates~\cite{mansdi,vedge-nsdi13,pathak2011,pathak2012,yoon} and in
+accurately attributing energy consumption, even for asynchronous and shared
+resources~\cite{cinder-eurosys11,osdi08-quanto}. Accurate energy models bring
+us closer to the goal of effective energy management on battery-constrained
+devices.
  
+But accurate energy measurement alone is not enough, because even
+perfectly-accurate measurements of energy consumption are insufficient to
+answer critical energy-related questions faced by users and developers,
+including:
+%
 \begin{itemize}
  
 \item Which of the following two apps is more energy efficient?
  
-\item Will this change to an apps code make it more energy efficient or not?
+\item Will this change to an app make it more energy efficient?
  
-\item Is a particular app an energy virus, and what does it even mean for an app
-to be an energy virus?
+\item Is a particular app an energy virus?
  
 \item How should the limited energy resources on a given app be prioritized?
  
 \end{itemize}
  
-What unifies all of these questions is one missing component: a measure of
-the \textit{value} delivered by an app, which can be used alone combined with
-energy consumption to compute energy \textit{efficiency} over any time
-interval:
+Unifying all of these questions is one missing component: a measure of
+app \textit{value}, which can be used alone or combined with
+energy consumption to compute energy \textit{efficiency}:
 %
 \[\frac{value}{energy} \]
 %
@@ -62,13 +55,12 @@ patterns. It must also work across a variety of different users with
 different usage patterns. It must be efficient to compute, since it should
 not compete for the same limited energy resources that it is intended to help
 manage. Ideally it should require little to no user input, since this will
-make it burdensome and error-prone. And to make matters worse, unlike
-measuring energy consumption there is no easy way to measure ground truth to
-compare against---even in the lab. Despite all these challenges, however,
-even a semi-accurate value measure would greatly benefit energy management on
-battery-constrained smartphones. Given that users have continued to report
-battery lifetimes as their top concern with today's
-models~\cite{jdpowerbatterylife-url}, we believe that this effort is
+make it burdensome and error-prone. And to make matters worse, there is no
+obvious way to measure ground truth to compare against---even in the lab.
+Despite all these challenges, however, even a semi-accurate value measure
+would greatly benefit energy management on battery-constrained smartphones.
+With users continuing to report battery lifetime as their top concern with
+smartphones~\cite{jdpowerbatterylife-url}, we believe this effort is
 worthwhile.
  
 In this paper we motivate the idea of a value measure and describe an early
@@ -119,15 +119,30 @@ redraws.
  
 \end{figure*}
  
-To evaluate our efficiency metric against usage based metric, we sent out a
-survey to our participants asking to answer if they would remove the 3 top
-energy inefficient apps suggested by both metrics to improve the battery-life
-of the smartphones by choosing one of the three options: yes, may be, no 47
-participants responded to the survey. Figure~\ref{fig-survey} shows that our
-efficiency metric did not do a better job than the usage based metric. This
-negative result points out that our content metric design is too simplistic
-to be effective. Only screen time or audio time is not enough to evaluate the
-different types of rich content delivered by apps. For example, our metric
-cannot distinguish between video content and interactive content. We also
-need to be careful about how we assign weight to the multiple components that
-consume energy to deliver the content.
+To continue the evaluation of our simple content-based value measure, we
+prepared a survey for the 107~\PhoneLab{} participants who contributed data
+to our experiment. Our goal was to determine if users would be more willing
+to remove inefficienct apps, as defined using our content-based measure. As a
+baseline, we also asked users about the apps that consumed the most energy.
+We used each participants data to generate a custom survey containing
+questions about 9 apps: the 3 least efficient apps as computed by our
+content-based value measure, the 3 apps that used the most energy on their
+smartphone during the experiment, and 3 apps chosen at random. For each we
+asked them a simple question: ``If it would improve your battery life, would
+you uninstall or stop using this app?'' To compute an aggregate score for
+both the content-based and usage based measures, we give each measure 1~point
+for a ``Yes'', 0.5~points for a ``Maybe'' and 0~points for a ``No''.
+47~participants completed the survey, and the results are shown in
+Figure~\ref{fig-survey}.
+
+Overall the results are inconclusive, with the content-delivery measure not
+clearly outperforming the straw-man usage measure at predicting which apps
+each user would be willing to remove to save battery life. Given the crude
+nature of our metric, this is not particularly surprising, and can be
+interpreted as a clear sign that we need a more sophisticated value measure
+incorporating several of the potential inputs we have previously discussed.
+However, on one level the results are very encouraging: most users were
+willing to consider removing one or more apps if that app would improve their
+battery lifetime. Clearly users are making this decision based on some idea
+of each app's value---the challenge is to replicate their choices using the
+information we have available to us.