blue / papers.hotmobile2015-numerator · Files

\section{Uses and Requirements}
\label{sec-usage}

To motivate the need for a value measure, we return to the questions posed in
the introduction. We explore each in more depth, providing a more detailed
description of how a value measure could be used. These use cases also help
us develop requirements for our metric, which are summarized at the end of
this section. We begin by exploring the basic question at the heart of the
problem: what is the value delivered by an app?

\subsection{What is Value?}

\begin{comment}

\begin{quote}
I shall not today attempt further to define the kinds of material I
understand to be embraced within that shorthand description; and perhaps I
could never succeed in intelligibly doing so. But I know it when I see it,
and the motion picture involved in this case is not that.\\
---Justice Potter Stewart~\cite{jacobellisvohio}.
\end{quote}

\end{comment}

All smartphone users intuitively realize that smartphone apps differ in
value---an email client, for example, is probably more valuable than a app
that makes farting sounds, and a chat client is probably more valuable than a
chat client that only sends the message ``yo''. In Supreme Court Justice
Potter Steward's famous formulation, they know value when they see it. But is
it possible to quantify these subjective distinctions and produce a value
measure?

To argue that this is possible we present two experiments that elucidate
smartphone app value in the form of both ordinal and cardinal utilities,
respectively:
%
\begin{enumerate}

\item An advisory will require you to remove some number of apps from your
smartphone. Order the apps you are currently using from least important to
most important. The N least important apps will be removed.

\item Your smartphone will require you to create an energy budget for the
apps you use. During any discharging cycle, once an app runs out of energy quota
you will not be able to use it until you plug in your smartphone. Allocate
battery percentages to each app you use.

\end{enumerate}
%
We plan to engage smartphone users in studies to explore in more detail which
of these approaches is more effective, comparing them by comparing users'
levels of satisfaction under each scenario. As an alternate formulation of
the second experiment to make it more similar to the first, the adversary
could remove the apps consuming the least energy up to a given target.

For our value measure we are hopeful that users will prove capable of
assigning cardinal utilities to apps---as in the second experiment---since
this matches most directly with our proposed value measure and could provide
ground truth for a value measure computed automatically. The second
experiment also engages users directly in the task of allocating energy,
which is one way that a value measure could be used. However, if ordinal
utilities prove more intuitive we can still compare the ordering generated by
our value metric with the ordering generated by users, although the specific
values of the measure will still require justification.

In either case, we believe that these experiments do suggest the existence of
quantifiable value for smartphone apps. We are not claiming, however, that
these setups are the only way or the right way to measure value. In both
cases low value measures have fairly extreme consequences---the app is
actually removed or rendered unusable. This may cause users to overvalue
essential tools such as communication apps and undervalue inessential apps
that nevertheless provide them with a great deal of enjoyment such as a game.
However, given that our goal is a value measure that can be paired with and
used to allocate energy, and that energy exhaustion has such severe
consequences on the usability of all apps, a more extreme experimental setup
may be justified.

\subsection{Comparing Apps}

With some confidence that smartphone app value can be quantified, we now
proceed to motivate the idea of a value measure by discussing several ways in
which it could be used.

The most powerful use of a value measure would be to compare apps by
comparing their energy efficiency, therefore overcoming the most critical
flaw in current attempts to compare or categorize apps by their energy
consumption alone~\cite{carat-sensys13}. Consider attempting to compare a
chat client and video-conferencing app by only measuring their energy
consumption. Unless it is terribly written, the chat client will consume less
energy. But this does not mean that it is efficient, or that the
video-conferencing app is not. Ultimately, all the energy consumption
comparison truly reveals is that the two apps do different things---which we
knew.

Using energy consumption alone even makes apples-to-apples comparison of the same app
difficult. Given an app that consumes twice as much energy on Alice's
smartphone than on Bob's, the question of why is left unanswered by pure
energy measures. Even if usage time can be used to normalize the comparison
to the power consumed, power consumption alone cannot incorporate differences
due to the different app features used by Alice and Bob or the different ways
that they have configured the app.

By computing value and, thus, energy efficiency, we can overcome these
weaknesses. A value measure should allow us to compare the efficiency of two
apps in different categories based on how efficiently they use energy to
deliver user value, making it possible to compare games to email clients to
video players. Comparisons within the same app category should allow users to
select the most efficient email client or web browser. Aggregating results
over all users, differences in app energy efficiency should reflect how well
the app is written and how well it predicts and adapts to users, not just
differences in the core features it provides. And when comparing two users
using the same app, differences in energy efficiency should reflect different
configurations or differences in how efficiently the app provides certain
features, not just that one user is using one feature and another is not.

\subsection{Evaluating App Changes}

A second use for the value measure is helping developers improve their apps
and deliver more value per joule. Today's energy profiling tools may be able
to show the energy impact of adding a new feature or changing the way that a
particular feature is implemented, but energy consumption alone is not
sufficient to apply Amdahl's Law properly to the problem of improving app
energy efficiency. For example, if a particular feature consumes a great deal
of energy but adds little value, it is possible that it should be eliminated,
not improved. Overall developers should strive to make the parts of their app
that generate a large amount of value as energy-efficient as possible, remove
parts that generate little value while consuming a great deal of energy, and
defer work on everything else.

\subsection{Detecting Energy Viruses}

A measure of app value makes it possible to produce a rigorous definition of
the term \textit{energy virus}: an app that produces little to no value per
joule. The choice of threshold will require some study, as it is unlikely
impossible to produce a single efficiency cut-off that cleanly separates
malicious apps from ones that are merely poorly-written. Note also that this
definition of energy virus can be made on a per-user basis. This is important
since a non-malicious but poorly-written app that continues to consume energy
even long after the user has stopped using it---and it has stopped providing
value---functions as an energy virus for that user, but may not for a user
that interacts with it more frequently.

\subsection{Prioritizing System Resources}

An app value measure should able to able to be used to prioritize limited
system resources, particularly energy but also storage, memory, networking
bandwidth and processor time. While mechanisms differ, most previous attempts
to control energy consumption rely on some form of rate control which
allocates a rate to each app and enforces that rate by slowing or stopping
the app when it exceeds its
allocation~\cite{odyssey-osr99,cinder-eurosys11,Zeng:2003,pixie-sensys08}.
However, all of these previous efforts have ignored the critical question of
\textbf{how rates should be set}. No matter how effective the enforcement
mechanisms are, systems that rely on rates will fail if they provide the same
rate to Skype and Snapchat, or to a very efficient app and an energy virus.

A measure of value can be used alone or in conjunction with energy consumption
to help prioritize limited energy resources. The simplest approach is to
attempt to enforce an energy allocation based on the relative value assigned
to each app. To encourage apps to be more energy efficient, it may also be
beneficial to weight allocations by their energy efficiency, providing a
boost to apps that provide a larger amount of value per joule. While there
are likely many ways to combine energy consumption with a value measure in
order to prioritize energy consumption, it is not clear that energy
consumption can be prioritized effectively without some measure of value. The
same approach can also be applied to determine how much of any limited system
resource to allocate to each app, with high-value apps gaining priority over
the processor, memory allocation, networking bandwidth and limited storage.
Together these resources allocation measures are designed to ensure that
high-value apps run smoothly at the expense of lower-value apps.

\subsection{Summary of Requirements}

The uses cases above give rise to a set of requirements for a possible value
measurement:
%
\begin{itemize}

\item It should enable aggregate comparisons between apps across categories and
users.

\item It should enable comparisons between the same app across users or
inputs, requiring that it be calculable given data from a single user.

\item It should enable targeted development by highlighting what parts of an
app generate value and what parts do not.

\item It should be able to be efficiently computed to not overly consume the
very resource that it is designed to help manage.

\item It should be derived with little to no input from the user.

\end{itemize}
%
We continue by discussing possible inputs to such a metric.