Commit 80c38ae52809962173d3c635ced9249ba0c9110b
minor edits
Showing
5 changed files
with
43 additions
and
67 deletions
figures/tables/tableALL.tex
| @@ -142,7 +142,7 @@ | @@ -142,7 +142,7 @@ | ||
| 142 | 142 | ||
| 143 | \caption{\small \textbf{Evaluating Components of a Value Measure.} | 143 | \caption{\small \textbf{Evaluating Components of a Value Measure.} |
| 144 | \PhoneLab{} data is used to weight overall app energy usage in a variety of | 144 | \PhoneLab{} data is used to weight overall app energy usage in a variety of |
| 145 | -different ways. Ommitted results are caused by Android reporting energy | 145 | +different ways. Omitted results are caused by Android reporting energy |
| 146 | consumption for non-apps such as the Android System.} | 146 | consumption for non-apps such as the Android System.} |
| 147 | 147 | ||
| 148 | \label{table-results} | 148 | \label{table-results} |
introduction.tex
| @@ -64,13 +64,12 @@ smartphones~\cite{jdpowerbatterylife-url}, we believe this effort is | @@ -64,13 +64,12 @@ smartphones~\cite{jdpowerbatterylife-url}, we believe this effort is | ||
| 64 | worthwhile. | 64 | worthwhile. |
| 65 | 65 | ||
| 66 | In this paper we motivate the idea of a value measure and describe an early | 66 | In this paper we motivate the idea of a value measure and describe an early |
| 67 | -failure at developing one based on measuring content delivery. We begin in | ||
| 68 | -Section~\ref{sec-usage} by describing how useful such a measure would be and | ||
| 69 | -the results of being able to answer questions like those posed above, as well | ||
| 70 | -as formulating design requirements for the value measure itself. | ||
| 71 | -Section~\ref{sec-measure} presents an overview of possible inputs into such a | ||
| 72 | -measure and discussion of how each could be measured and how useful it might | ||
| 73 | -be. In Section~\ref{sec-results} we present at formulating a value measure | ||
| 74 | -based on content delivered through the video display and audio output---an | ||
| 75 | -attempt that we consider a failure based on the result of a user survey, but | ||
| 76 | -one that we hope sheds light on how difficult this challenge may be. | 67 | +failure at developing one. We begin in Section~\ref{sec-usage} by describing |
| 68 | +how useful such a measure would be while also formulating design requirements | ||
| 69 | +for the value measure itself. Section~\ref{sec-measure} presents an overview | ||
| 70 | +of possible inputs into such a measure and discussion of how each could be | ||
| 71 | +measured and how useful it might be. In Section~\ref{sec-results} we present | ||
| 72 | +at formulating a value measure based on content delivered through the video | ||
| 73 | +display and audio output---an attempt that we consider a failure based on the | ||
| 74 | +result of a user survey, but a failure that we hope sheds some light on this | ||
| 75 | +difficult challenge. |
metric.tex
| @@ -26,7 +26,7 @@ foreground information---such as a pedometer app. | @@ -26,7 +26,7 @@ foreground information---such as a pedometer app. | ||
| 26 | 26 | ||
| 27 | Patterns of interaction may also be useful to observe, and inputs such as | 27 | Patterns of interaction may also be useful to observe, and inputs such as |
| 28 | keystrokes and touchscreen events are simple to track. However, there is more | 28 | keystrokes and touchscreen events are simple to track. However, there is more |
| 29 | -obvious differentation between app interaction patterns between | 29 | +obvious differentiation between app interaction patterns between |
| 30 | categories---users deliver far more keystrokes to a chat client than to a | 30 | categories---users deliver far more keystrokes to a chat client than to a |
| 31 | video player---so it is clear that interaction statistics will have to be | 31 | video player---so it is clear that interaction statistics will have to be |
| 32 | used in conjunction with complementary value measure components that offset | 32 | used in conjunction with complementary value measure components that offset |
results.tex
| @@ -112,7 +112,7 @@ redraws. | @@ -112,7 +112,7 @@ redraws. | ||
| 112 | \centering | 112 | \centering |
| 113 | \includegraphics[width=\textwidth]{./figures/survey.pdf} | 113 | \includegraphics[width=\textwidth]{./figures/survey.pdf} |
| 114 | 114 | ||
| 115 | -\caption{\textbf{Participant responses to energy inefficient app sugestions.} The height of each bar | 115 | +\caption{\textbf{Participant responses to energy inefficient app suggestions.} The height of each bar |
| 116 | demonstrates how many of the suggested apps the user is willing to remove for better battery life. } | 116 | demonstrates how many of the suggested apps the user is willing to remove for better battery life. } |
| 117 | 117 | ||
| 118 | \label{fig-survey} | 118 | \label{fig-survey} |
| @@ -122,7 +122,7 @@ redraws. | @@ -122,7 +122,7 @@ redraws. | ||
| 122 | To continue the evaluation of our simple content-based value measure, we | 122 | To continue the evaluation of our simple content-based value measure, we |
| 123 | prepared a survey for the 107~\PhoneLab{} participants who contributed data | 123 | prepared a survey for the 107~\PhoneLab{} participants who contributed data |
| 124 | to our experiment. Our goal was to determine if users would be more willing | 124 | to our experiment. Our goal was to determine if users would be more willing |
| 125 | -to remove inefficienct apps, as defined using our content-based measure. As a | 125 | +to remove inefficient apps, as defined using our content-based measure. As a |
| 126 | baseline, we also asked users about the apps that consumed the most energy. | 126 | baseline, we also asked users about the apps that consumed the most energy. |
| 127 | We used each participants data to generate a custom survey containing | 127 | We used each participants data to generate a custom survey containing |
| 128 | questions about 9 apps: the 3 least efficient apps as computed by our | 128 | questions about 9 apps: the 3 least efficient apps as computed by our |
usage.tex
| @@ -2,33 +2,17 @@ | @@ -2,33 +2,17 @@ | ||
| 2 | \label{sec-usage} | 2 | \label{sec-usage} |
| 3 | 3 | ||
| 4 | To motivate the need for a value measure, we return to the questions posed in | 4 | To motivate the need for a value measure, we return to the questions posed in |
| 5 | -the introduction. We explore each in more depth, providing a more detailed | ||
| 6 | -description of how a value measure could be used. These use cases also help | ||
| 7 | -us develop requirements for our metric, which are summarized at the end of | ||
| 8 | -this section. We begin by exploring the basic question at the heart of the | ||
| 9 | -problem: what is the value delivered by an app? | 5 | +the introduction and explore each in more depth. These use cases also help us |
| 6 | +develop requirements for our measure, which are summarized at the end of this | ||
| 7 | +section. We begin by exploring the basic question at the heart of the | ||
| 8 | +problem: what is the value of an app? | ||
| 10 | 9 | ||
| 11 | -\subsection{What is Value?} | ||
| 12 | - | ||
| 13 | -\begin{comment} | ||
| 14 | - | ||
| 15 | -\begin{quote} | ||
| 16 | -I shall not today attempt further to define the kinds of material I | ||
| 17 | -understand to be embraced within that shorthand description; and perhaps I | ||
| 18 | -could never succeed in intelligibly doing so. But I know it when I see it, | ||
| 19 | -and the motion picture involved in this case is not that.\\ | ||
| 20 | ----Justice Potter Stewart~\cite{jacobellisvohio}. | ||
| 21 | -\end{quote} | ||
| 22 | - | ||
| 23 | -\end{comment} | 10 | +\subsection{What is App Value?} |
| 24 | 11 | ||
| 25 | All smartphone users intuitively realize that smartphone apps differ in | 12 | All smartphone users intuitively realize that smartphone apps differ in |
| 26 | value---an email client, for example, is probably more valuable than a app | 13 | value---an email client, for example, is probably more valuable than a app |
| 27 | -that makes farting sounds, and a chat client is probably more valuable than a | ||
| 28 | -chat client that only sends the message ``yo''. In Supreme Court Justice | ||
| 29 | -Potter Steward's famous formulation, they know value when they see it. But is | ||
| 30 | -it possible to quantify these subjective distinctions and produce a value | ||
| 31 | -measure? | 14 | +that makes farting sounds. But is it possible to quantify these subjective |
| 15 | +distinctions and produce a value measure? | ||
| 32 | 16 | ||
| 33 | To argue that this is possible we present two experiments that elucidate | 17 | To argue that this is possible we present two experiments that elucidate |
| 34 | smartphone app value in the form of both ordinal and cardinal utilities, | 18 | smartphone app value in the form of both ordinal and cardinal utilities, |
| @@ -36,12 +20,12 @@ respectively: | @@ -36,12 +20,12 @@ respectively: | ||
| 36 | % | 20 | % |
| 37 | \begin{enumerate} | 21 | \begin{enumerate} |
| 38 | 22 | ||
| 39 | -\item An advisory will require you to remove some number of apps from your | 23 | +\item An adversary will require you to remove some number of apps from your |
| 40 | smartphone. Order the apps you are currently using from least important to | 24 | smartphone. Order the apps you are currently using from least important to |
| 41 | most important. The N least important apps will be removed. | 25 | most important. The N least important apps will be removed. |
| 42 | 26 | ||
| 43 | \item Your smartphone will require you to create an energy budget for the | 27 | \item Your smartphone will require you to create an energy budget for the |
| 44 | -apps you use. During any discharging cycle, once an app runs out of energy quota | 28 | +apps you use. During any discharging cycle, once an app runs out of energy |
| 45 | you will not be able to use it until you plug in your smartphone. Allocate | 29 | you will not be able to use it until you plug in your smartphone. Allocate |
| 46 | battery percentages to each app you use. | 30 | battery percentages to each app you use. |
| 47 | 31 | ||
| @@ -49,19 +33,15 @@ battery percentages to each app you use. | @@ -49,19 +33,15 @@ battery percentages to each app you use. | ||
| 49 | % | 33 | % |
| 50 | We plan to engage smartphone users in studies to explore in more detail which | 34 | We plan to engage smartphone users in studies to explore in more detail which |
| 51 | of these approaches is more effective, comparing them by comparing users' | 35 | of these approaches is more effective, comparing them by comparing users' |
| 52 | -levels of satisfaction under each scenario. As an alternate formulation of | ||
| 53 | -the second experiment to make it more similar to the first, the adversary | ||
| 54 | -could remove the apps consuming the least energy up to a given target. | ||
| 55 | - | ||
| 56 | -For our value measure we are hopeful that users will prove capable of | ||
| 57 | -assigning cardinal utilities to apps---as in the second experiment---since | ||
| 58 | -this matches most directly with our proposed value measure and could provide | ||
| 59 | -ground truth for a value measure computed automatically. The second | ||
| 60 | -experiment also engages users directly in the task of allocating energy, | ||
| 61 | -which is one way that a value measure could be used. However, if ordinal | ||
| 62 | -utilities prove more intuitive we can still compare the ordering generated by | ||
| 63 | -our value metric with the ordering generated by users, although the specific | ||
| 64 | -values of the measure will still require justification. | 36 | +levels of satisfaction under each scenario. For our value measure we are |
| 37 | +hopeful that users will prove capable of assigning cardinal utilities to | ||
| 38 | +apps---as in the second experiment---since this matches most directly with | ||
| 39 | +our proposed value measure and could provide ground truth for a value measure | ||
| 40 | +computed automatically. The second experiment also engages users directly in | ||
| 41 | +the task of allocating energy, which is one way that a value measure could be | ||
| 42 | +used. However, if ordinal utilities prove more intuitive we can still compare | ||
| 43 | +the ordering generated by our measure with the ordering generated by users, | ||
| 44 | +although the values of the measure will still require justification. | ||
| 65 | 45 | ||
| 66 | In either case, we believe that these experiments do suggest the existence of | 46 | In either case, we believe that these experiments do suggest the existence of |
| 67 | quantifiable value for smartphone apps. We are not claiming, however, that | 47 | quantifiable value for smartphone apps. We are not claiming, however, that |
| @@ -85,20 +65,19 @@ The most powerful use of a value measure would be to compare apps by | @@ -85,20 +65,19 @@ The most powerful use of a value measure would be to compare apps by | ||
| 85 | comparing their energy efficiency, therefore overcoming the most critical | 65 | comparing their energy efficiency, therefore overcoming the most critical |
| 86 | flaw in current attempts to compare or categorize apps by their energy | 66 | flaw in current attempts to compare or categorize apps by their energy |
| 87 | consumption alone~\cite{carat-sensys13}. Consider attempting to compare a | 67 | consumption alone~\cite{carat-sensys13}. Consider attempting to compare a |
| 88 | -chat client and video-conferencing app by only measuring their energy | 68 | +chat client and videoconferencing app by only measuring their energy |
| 89 | consumption. Unless it is terribly written, the chat client will consume less | 69 | consumption. Unless it is terribly written, the chat client will consume less |
| 90 | energy. But this does not mean that it is efficient, or that the | 70 | energy. But this does not mean that it is efficient, or that the |
| 91 | -video-conferencing app is not. Ultimately, all the energy consumption | 71 | +videoconferencing app is not. Ultimately, all the energy consumption |
| 92 | comparison truly reveals is that the two apps do different things---which we | 72 | comparison truly reveals is that the two apps do different things---which we |
| 93 | -knew. | 73 | +already knew. |
| 94 | 74 | ||
| 95 | Using energy consumption alone even makes apples-to-apples comparison of the same app | 75 | Using energy consumption alone even makes apples-to-apples comparison of the same app |
| 96 | difficult. Given an app that consumes twice as much energy on Alice's | 76 | difficult. Given an app that consumes twice as much energy on Alice's |
| 97 | smartphone than on Bob's, the question of why is left unanswered by pure | 77 | smartphone than on Bob's, the question of why is left unanswered by pure |
| 98 | energy measures. Even if usage time can be used to normalize the comparison | 78 | energy measures. Even if usage time can be used to normalize the comparison |
| 99 | to the power consumed, power consumption alone cannot incorporate differences | 79 | to the power consumed, power consumption alone cannot incorporate differences |
| 100 | -due to the different app features used by Alice and Bob or the different ways | ||
| 101 | -that they have configured the app. | 80 | +due to the different app features or configurations used by Alice and Bob. |
| 102 | 81 | ||
| 103 | By computing value and, thus, energy efficiency, we can overcome these | 82 | By computing value and, thus, energy efficiency, we can overcome these |
| 104 | weaknesses. A value measure should allow us to compare the efficiency of two | 83 | weaknesses. A value measure should allow us to compare the efficiency of two |
| @@ -108,10 +87,10 @@ video players. Comparisons within the same app category should allow users to | @@ -108,10 +87,10 @@ video players. Comparisons within the same app category should allow users to | ||
| 108 | select the most efficient email client or web browser. Aggregating results | 87 | select the most efficient email client or web browser. Aggregating results |
| 109 | over all users, differences in app energy efficiency should reflect how well | 88 | over all users, differences in app energy efficiency should reflect how well |
| 110 | the app is written and how well it predicts and adapts to users, not just | 89 | the app is written and how well it predicts and adapts to users, not just |
| 111 | -differences in the core features it provides. And when comparing two users | ||
| 112 | -using the same app, differences in energy efficiency should reflect different | 90 | +differences in the core features it provides. When comparing two users using |
| 91 | +the same app, differences in efficiency should reflect different | ||
| 113 | configurations or differences in how efficiently the app provides certain | 92 | configurations or differences in how efficiently the app provides certain |
| 114 | -features, not just that one user is using one feature and another is not. | 93 | +features. |
| 115 | 94 | ||
| 116 | \subsection{Evaluating App Changes} | 95 | \subsection{Evaluating App Changes} |
| 117 | 96 | ||
| @@ -132,7 +111,7 @@ defer work on everything else. | @@ -132,7 +111,7 @@ defer work on everything else. | ||
| 132 | A measure of app value makes it possible to produce a rigorous definition of | 111 | A measure of app value makes it possible to produce a rigorous definition of |
| 133 | the term \textit{energy virus}: an app that produces little to no value per | 112 | the term \textit{energy virus}: an app that produces little to no value per |
| 134 | joule. The choice of threshold will require some study, as it is unlikely | 113 | joule. The choice of threshold will require some study, as it is unlikely |
| 135 | -impossible to produce a single efficiency cut-off that cleanly separates | 114 | +impossible to produce a single efficiency cutoff that cleanly separates |
| 136 | malicious apps from ones that are merely poorly-written. Note also that this | 115 | malicious apps from ones that are merely poorly-written. Note also that this |
| 137 | definition of energy virus can be made on a per-user basis. This is important | 116 | definition of energy virus can be made on a per-user basis. This is important |
| 138 | since a non-malicious but poorly-written app that continues to consume energy | 117 | since a non-malicious but poorly-written app that continues to consume energy |
| @@ -150,9 +129,9 @@ allocates a rate to each app and enforces that rate by slowing or stopping | @@ -150,9 +129,9 @@ allocates a rate to each app and enforces that rate by slowing or stopping | ||
| 150 | the app when it exceeds its | 129 | the app when it exceeds its |
| 151 | allocation~\cite{odyssey-osr99,cinder-eurosys11,Zeng:2003,pixie-sensys08}. | 130 | allocation~\cite{odyssey-osr99,cinder-eurosys11,Zeng:2003,pixie-sensys08}. |
| 152 | However, all of these previous efforts have ignored the critical question of | 131 | However, all of these previous efforts have ignored the critical question of |
| 153 | -\textbf{how rates should be set}. No matter how effective the enforcement | ||
| 154 | -mechanisms are, systems that rely on rates will fail if they provide the same | ||
| 155 | -rate to Skype and Snapchat, or to a very efficient app and an energy virus. | 132 | +how rates should be set. No matter how effective the enforcement mechanisms |
| 133 | +are, systems that rely on rates will fail if they provide the same rate to | ||
| 134 | +Skype and Snapchat, or to a very efficient app and an energy virus. | ||
| 156 | 135 | ||
| 157 | A measure of value can be used alone or in conjunction with energy consumption | 136 | A measure of value can be used alone or in conjunction with energy consumption |
| 158 | to help prioritize limited energy resources. The simplest approach is to | 137 | to help prioritize limited energy resources. The simplest approach is to |
| @@ -186,10 +165,8 @@ inputs, requiring that it be calculable given data from a single user. | @@ -186,10 +165,8 @@ inputs, requiring that it be calculable given data from a single user. | ||
| 186 | app generate value and what parts do not. | 165 | app generate value and what parts do not. |
| 187 | 166 | ||
| 188 | \item It should be able to be efficiently computed to not overly consume the | 167 | \item It should be able to be efficiently computed to not overly consume the |
| 189 | -very resource that it is designed to help manage. | 168 | +resources that it is designed to help manage. |
| 190 | 169 | ||
| 191 | \item It should be derived with little to no input from the user. | 170 | \item It should be derived with little to no input from the user. |
| 192 | 171 | ||
| 193 | \end{itemize} | 172 | \end{itemize} |
| 194 | -% | ||
| 195 | -We continue by discussing possible inputs to such a metric. |