New.

Geoffrey Challen
1 parent fc352ee3
Showing 4 changed files with 83 additions and 100 deletions
certainty.tex
conclusion.tex
discussion.tex
related.tex
@@ -17,58 +17,44 @@ variety of different adaptation patterns.
 The optional \texttt{evaluate} block of a \texttt{maybe} statement allows
 programmers to provide app-specific \textit{a posteriori} evaluation logic.
 However, in many cases, we expect that \texttt{maybe} statements will be used
-to achieve common objectives such as improving performance or saving energy.
-To streamline application development, our current system evaluates
-\texttt{maybe} statements without a \texttt{evaluate} block by measuring both
-energy and performance. In cases where one alternative optimizes both, that
-alternative will be used---although the decision may still be time-varying
-due to dependence on time-varying factors such as network availability.
-
-\sloppypar{Cases where \texttt{maybe} statements provide an
-energy-performance tradeoff require more investigation. We are exploring
-several options, including collapsing both metrics into a single score by
-computing the energy-delay product (EDP) of each alternative, or allowing
-users to set a per-app preference indicating whether energy or performance
-should determine \texttt{maybe} alternative selection.}
-
-When \texttt{maybe} statements include a \texttt{evaluate} block, their
-custom evaluation logic must merge any relevant factors into a single score,
-although \texttt{evaluate} blocks can still record other relevant information
-to aid in understanding app behavior. Specifically, while the \texttt{score}
-value is used to evaluate the alternative, the entire JSON object returned by
-the \texttt{evaluate} block is recorded for later analysis. Our standard
-energy and performance \texttt{evaluate} block records energy and performance
-separately in the JSON object but combines them as required into a single
-score. Note that because the \texttt{evaluate} block delivers information to
-the developer through the \texttt{maybe} service, as described below,
-\texttt{maybe} statements can be connected with end-to-end app performance
-metrics that would normally not be visible on the device. \texttt{evaluate}
-blocks may also want to query the user directly, and we are exploring ways to
-make this possible.
-
-In some cases we expect that \texttt{evaluate} blocks may need to know which
-alternative was executed to determine the score---for example, if the two
-alternatives produce different quality output. The simplest solution is to
-have each alternative set a variable indicating that it was used, but we
-believe that eventually including some form of labeling syntax for
-\texttt{maybe} alternatives may be beneficial.
+to achieve common objectives such as improving performance or saving energy. To
+streamline application development, our current system evaluates \texttt{maybe}
+statements without a \texttt{evaluate} block by measuring both energy and
+performance. In cases where one alternative optimizes both, that alternative
+will be used---although the decision may still be time-varying due to
+dependence on time-varying factors such as network availability. When
+alternatives produce an energy-performance tradeoff we are exploring several
+options, including collapsing both metrics into a single score by computing the
+energy-delay product (EDP) of each alternative, or allowing users to set a
+per-app energy or performance preference.
+
+\texttt{evaluate} blocks can also record other information to aid adaptation.
+While the \texttt{score} value is used to evaluate the alternative, the
+entire JSON object returned by the \texttt{evaluate} block is delivered to
+the developer for later analysis. This allows \texttt{maybe} statements to be
+connected with end-to-end app performance metrics not visible on the device.
+\texttt{evaluate} blocks may also want to query the user directly, and we are
+exploring ways to make this possible.
+
+We expect that some \texttt{evaluate} blocks may need to know which
+alternative was executed to compute a score---for example, if the two
+alternatives produce different quality output. We are exploring the use of
+automatically-generated labels to aid this process.
 If a \texttt{maybe} alternative throws an error, the system will bypass the
 \texttt{evaluate} block and give it the worst possible score. By integrating
-a form of record-and-replay~\cite{gomez2013reran}, it may also be possible to
-roll back the failed alternatives changes and retry another alternative.
-\texttt{maybe} is intended to enable adaptation, not avoid errors, but the
-existence of other alternatives provides our system with a way to work around
-failures caused by uncertainty. Resistance to errors may also encourage
-developers to use \texttt{maybe} statements to prototype alternatives to
-existing well-tested code.
+a form of record-and-replay~\cite{gomez2013reran}, it may be possible to roll
+back the failed alternative and retry another. \texttt{maybe} is intended to
+enable adaptation, not avoid errors, but the existence of other alternatives
+provides a way to work around failures caused by uncertainty. Fault tolerance
+may also encourage developers to use \texttt{maybe} statements to prototype
+alternatives to existing well-tested code.
 A final question concerns when a \texttt{maybe} alternative should be
-evaluated. In some cases it may be appropriate for \textit{a posteriori}
-evaluation to happen immediately after execution. In others, it may be
-necessary to continue executing the same alternative over a period of time to
-perform a fair comparison. As described previously, \texttt{evaluate} blocks
-can indicate explicitly whether or not to continue evaluating the
+evaluated. Some alternatives may require evaluation immediately after
+execution. Others may require repeated execution over a longer period of time
+to perform a fair comparison. As described previously, \texttt{evaluate}
+blocks can indicate explicitly whether or not to continue evaluating the
 alternative, and we are determining how to make a similar choice available to
 \texttt{maybe} statements without \texttt{evaluate} blocks. In addition,
 \texttt{evaluate} blocks can store state across multiple alternative
-\vspace*{-0.05in}
 \section{Conclusion}
 \label{sec-conclusion}
@@ -8,7 +7,6 @@ development time and for that uncertainty to be resolved through later
 testing and adaptation. We are in the process of building a prototype of the
 \texttt{maybe} system for Android smartphones.
-\vspace*{-0.05in}
 \section*{Acknowledgments}
 Students and faculty working on the \texttt{maybe} project are supported by
@@ -19,5 +17,3 @@ and
 \href{http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=1423215}{1423215}.
 The \texttt{maybe} team thanks the anonymous reviewers and our shepherd,
 Mahadev Satyanarayanan, for their feedback.
-
-\vspace*{-0.05in}
@@ -5,44 +5,33 @@
 We have yet to determine how natural programmers will find the \texttt{maybe}
 statement. Encouragingly, \texttt{maybe} statements are similar to the
-ubiquitous \texttt{if-else} statement, and in many cases can be used to
-directly replace \texttt{if-else} statements that attempt runtime adaptation.
-To coordinate the adaptation of multiple code paths a single \texttt{maybe}
-variable---such as the policy string in Figure~\ref{fig-maybeexamples}---can
-be used to control multiple \texttt{if-else} statements.
+ubiquitous \texttt{if-else} statement, and in many cases can directly replace
+\texttt{if-else} statements that attempt runtime adaptation. To coordinate
+the adaptation of multiple code paths a single \texttt{maybe} variable can be
+used to control multiple \texttt{if-else} statements.
-While \texttt{maybe} is a powerful programming tool, it is important that it
-be used sparingly. If objective dependencies exist between \texttt{maybe}
-statements, the overall configuration space may expand exponentially,
-complicating post-deployment adaptation process. Compile-time analysis may be
-required to detect dependencies between \texttt{maybe} statements and
-encourage programmers to limit their use of \texttt{maybe} if to ensure that
-downstream optimization remains feasible.
+Overuse of the \texttt{maybe} statement may cause problems. If dependencies
+exist between \texttt{maybe} statements, the overall configuration space may
+expand exponentially, complicating post-deployment adaptation. Compile-time
+analysis may be required to detect dependencies between \texttt{maybe}
+statements and encourage programmers to limit their use of \texttt{maybe} to
+ensure that downstream optimization remains feasible.
-\texttt{maybe} statements should not be used when adaptation is not
-app-specific and can be refactored into a library. As an example, an app
-should not use \texttt{maybe} to decide which network interface to use when
-attempting to achieve a common objective, such as maximizing throughput. This
-choice would be better refactored into a dedicated library, which might use
-its own internal \texttt{maybe} statements. Not only is the resulting
-codebase smaller, but the total number of \texttt{maybe} statements that the
-system has to test is reduced.
+\texttt{maybe} statements should not be used when adaptation can be
+refactored into a library. As an example, an app should not use
+\texttt{maybe} to decide which network interface to use when attempting to
+achieve a common objective, such as maximizing throughput. This adaptation
+should be refactored into a dedicated library, which might use its own
+\texttt{maybe} statements. Not only is the resulting codebase smaller, but
+the total number of \texttt{maybe} statements to test is reduced.
 However, the \texttt{maybe} statement represents a fundamentally different
-approach from attempts to enable runtime adaptation that rely on libraries.
-As a code organization strategy, the benefits that libraries provide in terms
-of reduced duplication and more powerful interfaces are orthogonal and
-complementary to the \texttt{maybe} statement. However, a large amount of
-code including adaptation logic remains app specific and cannot be refactored
-into a library.
-
-More importantly, library development still requires development-time
-certainty. Despite the fact that library developers are more likely to be
-experts at the type of adaptation the library performs, we still believe that
-even the most skilled programmers are not capable of anticipating all
-possible sources of uncertainty and will benefit from being able to express
-structured uncertainty. \texttt{maybe} allows all developers---including both
-app and library writers---to shed the burden of producing a single certain
-approach and instead write uncertain code containing the flexibility needed
-required to enable powerful data-driven approaches to post-deployment
-adaptation.
+approach to runtime adaptation than systems that rely on libraries because
+library development still requires development-time certainty. While library
+developers are more likely to be experts at the type of adaptation their
+library performs, we still believe that even the most skilled programmers
+will benefit from being able to express structured uncertainty.
+\texttt{maybe} allows all developers---including both app and library
+writers---to shed the burden of producing a single certain approach and
+instead write uncertain code containing the flexibility required to enable
+powerful data-driven approaches to post-deployment adaptation.
@@ -7,21 +7,20 @@ programming with uncertain data, rather than the runtime adaptation enabled
 by \texttt{maybe}.
 Aspect oriented programming (AOP)~\cite{aop} aims to increase modularity
-through the separation of cross-cutting concerns. The programmer can express
-cross-cutting concerns in stand alone modules, or aspects. Aspects are
-composed of advice, which specifies a computation to be performed as well as
-points in the program at which that computation should be performed.
-Fundamentally, the goals of AOP and the \texttt{maybe} statement differ, with
-AOP focusing on modularity and \texttt{maybe} focused on increasing runtime
-flexibility in the face of uncertainty.
+through the separation of cross-cutting concerns. The programmer expresses
+cross-cutting concerns in stand alone modules, or aspects, which specify a
+computation to be performed as well as points in the program at which that
+computation should be performed. Fundamentally, the goals of AOP and the
+\texttt{maybe} statement differ, with AOP focusing on modularity and
+\texttt{maybe} focused on enabling adaptation by expressing uncertainty.
-\texttt{maybe} shares similaries with language-based approaches to managing
+\texttt{maybe} shares similaries with language-based approaches to adapting
 energy consumption such as Eon~\cite{sensys07-eon} and
 Levels~\cite{sensys07-levels}. However, these approaches still require
-programmers to express certainty by associating code with particular energy
-states, rather than allowing the \texttt{maybe} system to determine which
-energy states are appropriate. \texttt{maybe} can also enable adaptation
-driven by goals other than energy management.
+programmers to express certainty by associating code with energy states,
+rather than allowing the \texttt{maybe} system to determine which energy
+states are appropriate. \texttt{maybe} can also enable adaptation driven by
+goals other than energy management.
 Attempts to enable more adaptive mobile systems date back to systems such as
 Odyssey~\cite{odyssey-sosp97}. However, a taxonomy of approaches to enabling
@@ -30,3 +29,16 @@ the focus of early efforts on incorporating adaptation into libraries that
 could be used by multiple apps. As we have pointed out previously, while
 adaptation libraries are useful, \texttt{maybe} statements can make them more
 powerful by allowing programmers to express uncertainty.
+
+Recent approaches that allow mobile devices to effectively offload
+computation by automating client-cloud partitioning are also related to the
+\texttt{maybe} statement. Systems such as Tactics~\cite{tactics-mobisys03}
+and MAUI~\cite{FIXME} used a variety of approaches to enabling this form of
+adaptation but are narrowly-focused on harnessing opportunities for remote
+execution. While \texttt{maybe} statements can enable simple decisions about
+whether to offload portions of an app, the \texttt{maybe} approach is not yet
+rich enough to consider how much of the execution should be performed
+remotely or deal with server failures. At present \texttt{maybe} focuses on
+single-device adaptation, but we are interested in exploring the ability to
+use uncertainty to distribute computation between multiple devices as future
+work.