% // vim: set ft=tex:
\chapter{Introduction}
\label{context::introduction}
This document contains a study on the feasibility of using compile-time code analysis, as found in \gls{Rust}'s \gls{compiler}, for ensuring memory-safety within an \gls{os} kernel.
This study could be applied to all \glspl{app}, but the focus is on the implementation of \glspl{os} which is the \gls{app} that is responsible for managing the system's resources and provide abstractions for all other \glspl{app}.
For this the \gls{os} is the only \gls{app} that required unrestricted access to these resources, with the responsibility of managing them safely according to the rules that are either hard-coded or set up by the \gls{sysadmin}.

The increasing number of vulnerabilities based on memory-safety issues in \glspl{app}, as presented in \cref{context::weaknesses-mem-safety::cwe::statistics}, is a major motivator for working on this topic.

\section{Motivational Hypothesis}
% Primary Research Questions
% The primary research question is the basis for data collection and arises from the Purpose of the Study. There may be one, or there may be several. When the research is finished, the contribution to the knowledge will be the answer to these questions. Do not confuse the primary research questions with interview questions in a qualitative study, or survey questions in a quantitative study. The research questions in a qualitative study are followed by both a null and an alternate hypothesis.
% Hypotheses
% A hypothesis is a testable prediction for an observed phenomenon, namely, the gap in the knowledge. Each research question will have both a null and an alternative hypothesis in a quantitative study. Qualitative studies do not have hypotheses. The two hypotheses should follow the research question upon which they are based. Hypotheses are testable predictions to the gap in the knowledge. In a qualitative study the hypotheses are replaced with the primary research questions.

According to my best-effort literature research in Q1/2017, the hypothesis that \textit{Rust's static code analysis can guarantee memory safety in the \gls{os}} has not been studied explicitly.
This is to my surprise, because as explained in \cref{context::introduction::memory-safety}, memory-safety in \gls{os} development is critical, and \gls{Rust} offers attractive features that might bring improvements, which is covered in \cref{rnd::rust}.
The hypothesis cannot be trivially approved or denied, which drives the research efforts for my final study project.

Besides this specific hypothesis, many implementations of \glspl{os} with \gls{Rust} have appeared in public.
Their purposes range from proof-of-concept and educational work like \gls{imezzos} and \gls{blogos}, to implementations that aim to be production grade software like \gls{redoxos} and \gls{tockos} \cite{Levy2015a}.
These implementations are subject to evaluation in \cref{rnd::existing-os-dev-with-rust}

The final results will be of qualitative nature, captured by analyzing the existing and a self-developed \gls{Rust}-implementations of popular memory management techniques.
In addition to the sole analysis of \gls{Rust}-implementations, comparisons will be made, discerning the level of memory safety guarantees gained over similarly intending implementations in \gls{C}.

\section{What is Memory-Safety?}
\label{context::introduction::memory-safety}
Memory-safety is a term that is only vaguely defined in general, thus a definition is given for the context of this study.  
For a thorough understanding of the issues discussed further in this document, it might be helpful to review the basics of how memory is used in current computer systems.

For decades computer systems, more specifically their \glspl{cpu}, were designed to execute instructions that were previously loaded into volatile main memory, typically from a secondary, persistent memory.
These instructions are themselves able to alter the very main memory they are stored at, which allows for great flexibility but also involves the risk of corrupting a consistent chain of instructions or other memory content like data.

As any other \gls{app}, the \gls{os} is loaded and executed in form of one or multiple sets of logically grouped instructions, called \glspl{program}. 
Loading the \gls{os}'s program into memory is not the responsibility of the \gls{os}, it belongs to the components earlier in the boot process, namely the boot loader and system firmware.
The \gls{os} takes over the responsibility to protect the main and secondary memory as soon as the bootloader has loaded the \gls{os} and has jumped to its first instruction.
From this point, loading further programs into main memory is done by the \gls{os}, either according to scheduled jobs set up by the \gls{sysadmin}, or based on well-defined events which can be triggered by any form of input via the system's interfaces.
For example, the \gls{os} can load and execute a program stored on the hard-disk, after the user has given the appropriate instructions via a terminal.

The execution of other programs is potentially dangerous, because they might attempt to access the memory content of other programs and their data.
It is the responsibility of the \gls{os} to prevent such executed programs from being able to mutually interfere with memory content that is not theirs, keeping the memory in a safe state at all times \footnote{This does not include memory-safety \textit{within} each of these executed programs, as the \gls{os} has no pertinent knowledge of the program's intentions.}.
This requires an extensive amount of care and foresight from the developers of the \gls{os}, to ensure memory consistency in any of the various events and combinations thereof that might possibly occur at runtime.

\subsection{A Definition Of Memory-Safety For \glsentryplural{os}}
\label{context::introduction::memory-safety::def}
If the \gls{os} is memory-safe, any program, whether it is part of the \gls{os} or any installed \gls{app}, is only able to access its allocated memory regions.
Additionally, if the \gls{os} supports shared memory regions, each shared memory region may only be accessible by programs that have been granted access to it.

\section{Memory-Safety Violation in Software}
\label{context::introduction::memory-safety-violation-in-sw}
Software that has memory-safety violations is vulnerable to random crashes and intentional attacks.
This is why information on safety related mistakes in software shouldn't be publicly available immediately.
Ideally, before the vulnerability is publicly known, all systems that run the erroneous software in question should have the chance to update the software is question, so that any potential attackers can't leverage the known vulnerability.
This introduces a dilemma, because software updates usually contain publicly known information, at least in the open-source sector.

Any existing or hypothetical solution to this dilemma is not in scope of this study, but two conclusions can be made.
First, public statistics in the area of software vulnerabilities are questionable with regard to their completeness.
Second, and more importantly, memory-safety related software mistakes should be detected as early as possible, ideally before the software is released and installed anywhere.

\subsection{Human Aspect}
\label{context::introduction::human-aspect}
To detect software mistakes early, it is helpful to analyze where they originate.
This section emphasizes the fact that software - even if software-generators are interleaved - is ultimately produced by humans.
This aspect is relevant to assessing the origins of memory-safety related errors, as only errors made by humans during any stage of the development process can lead to unsafe memory access at runtime.
The following assumptions are made based on common sense
\begin{itemize}
\item{No human is born as a flawless software engineer.}
\item{Beginners will start writing production software before they master programming in perfection.}
\item{With each generation of humans there will always be new beginners that will start learning from scratch.}
\item{Capabilities and motivation vary significantly between individuals.}
\item{Less capable or motivated individuals will eventually write software for production use.}
\item{Education is not ideal.}
\end{itemize}
Combining these assumptions, it cannot generally be assumed that every beginner that writes software has learned about the involved risks, and is determined and capable to ensure memory-safety and other high quality standards in their software.

From my personal experience with software developers and students of software engineering, I have received the impression that many do not prioritize safety in their software.

One severe example for this in my personal career so far is a former team partner in one of our \gls{C}/\gls{C++} programming courses.
Despite the fact that the professor instructed us to use valgrind\footnote{a runtime memory analyzer and debugger} to verify our programs, my partner was satisfied with the result after writing the algorithms to his best understanding and correcting all errors detected by the \gls{compiler}.
Discussing the topic with him did not lead to any understanding on his side, and even after verifying that his program had easily detectable memory issues, he insisted on the correct result of the algorithm and pointed out the lack of time.
I realized similar mindset in some of the other teams.

This personal experience is no scientific proof nor is it statistically significant.
It does create a feeling of insecurity, because if their software is distributed widely a few of these people are enough to risk the security of thousands of systems.

A professor and co-author of \citetitle{Arpaci-Dusseau2015} gives the following warning about this issue:
\textit{"Just because a program compiled(!) or even ran once or many times correctly does not mean the program is correct. Many events may have conspired to get you to a point where you believe it works, but then something changes and it stops. A common student reaction is to say (or yell) “But it worked before!” and then blame the compiler, operating system, hardware, or even (dare we say it) the professor. But the problem is usually right where you think it would be, in your code. Get to work and debug it before you blame those other components."}\cite[p.~127]{Arpaci-Dusseau2015}

Plenty of educational, economical or methodological solutions are imaginable for this problem.
Higher focus on safety and testing in education, enforced internal company guidelines, or industry wide third party software certification requirements can be attempted.
For this study such constraints are out of scope, and the focus is on examining technical methods that detect and indicate mistakes as early as possible.

\subsection{Technical Aspect}
\label{context::introduction::technical-aspect}
As established in the \nameref{context::introduction::human-aspect}, it can not be prevented for individuals to type erroneous code into their code editors, thus a technical solution must be found.
The problem on the technical side is that the \gls{compiler} is not able to detect all errors that are in the source code and humans are able to produce an executable program from problematic source code.
The resulting executable program might merely serve its purpose, and can contain severe technical mistakes that are not considered an error by the \gls{compiler}.
This is especially likely using low-abstraction languages like \gls{C} and \gls{C++} for \gls{os} development, where technical mistakes and intended behavior are very difficult to distinguish.

The ultimate goal beyond this project must be to prevent any kind of vulnerable or erroneous software from being distributed to production systems.
This project's contribution towards that goal is to evaluate if \gls{Rust}'s \gls{compiler} is able to guarantee safe memory-management in the \gls{os}.

\subsection{Time Aspect}
\label{context::introduction::memory-safety::time-aspect}
Tests are important for the functionality and safety of a \gls{app}.
In addition to the presence and quality of tests, their timing in the software life cycle plays an important role.
It is desirable to place software tests as early as possible in the software life cycle and detect mistakes then, rather than compromising production-systems that hold sensitive data and offer important services.

\begin{figure}[ht]
\centering
\smartdiagramset{
    back arrow disabled=true,
    module minimum width=5cm,
    text width=4cm,
    additions={
        additional item offset=1cm,
        additional item width=4cm,
        additional item border color=gray,
        additional item text width=4cm,
        additional arrow tip=to,
        additional arrow line width=2pt,
        additional arrow color=black,
    },
}
\smartdiagramadd[flow diagram]{
    Design,
    Programming,
    Compilation,
    Distribution,
    Execution
    }{%
    right of module2/When vulnerabilities are created,
    right of module3/When they should be detected,
    right of module5/When they create real problems
}
\smartdiagramconnect{to-}{module2/additional-module1}
\smartdiagramconnect{to-}{module3/additional-module2}
\smartdiagramconnect{to-}{module5/additional-module3}
\caption{Simple Software Life Cycle}
\label{fig:simple-software-lifecycle}
\end{figure}
A simple software life cycle model is visualized in \cref{fig:simple-software-lifecycle}.

Assuming that there is a software design, even a flawless one, the next step is to translate this flawless design into flawless source code.
The is a manual translation step from one representation into another, leaving much room for mistakes.
The next component in the chain is the \gls{compiler} which processes the source code by translating it to the next representation, machine code.
This is the step that where source code analysis can be technically enforced.
Ideally, as previously explained, the \gls{compiler} should be able to detect the programmers mistakes, especially the ones that have a negative impact on memory-safety.
Advanced programmers can profit too, as everybody makes mistakes from time to time, depending on the level of focus, which is not a constant.

When programmers talk about tests, they typically mean little functions they have to write to test the actual functions of their programs.
This kind of testing requires the execution of the program after its compilation, and is likely to neglect severe technical mistakes.
The method of testing the source code directly has much more potential, because relations between source code objects can be analyzed.
Various \textit{static code analyzer} tools have become popular for such analysis.
However, these are always optional and the developer cannot be forced to use them.
Much different is the situation when the analysis is part of the \gls{compiler}, which is the case in \gls{Rust}.

\chapter{OS Development: Concepts, Conventions, Risks}
\label{context::os-dev-concepts}
In order to protect the memory of each executed program according to \cref{context::introduction::memory-safety::def}, the \gls{os} must be designed developed, and tested carefully.
This chapter explains concepts used in \gls{os} development and points out memory-safety critical operations in necessary detail.
This is done in preparation for the next \cref{context::weaknesses-mem-safety}, which explains weaknesses that result from memory management mistakes that were made in the attempt to implement the following concepts.

Since the \gls{os} manages the system's hardware directly, many of the implementation and design choices depend on hardware design and architecture.
To bound the extent of this document, the explanations are limited to one contemporary architecture, \gls{amd64}, and further narrowed down by focusing on the operation in 64-Bit Long Mode\cite[p.~18]{AMD64Vol2}.

\section{Virtualization}
\label{context::os-dev-concepts::virtualization}
The \gls{os} is considered the lowest software layer on the system and must know the very details of the system's hardware resources and perform raw access to it.
The goal is to make \gls{os} the only software on the system that is required to have this particular knowledge, so that other \glspl{app} could run on virtually any system.

\subsection{Abstraction Layers}
The first step of \gls{virt}\footnote{The term \textit{virtualization} within the \gls{os} the jargon can be understood as abstraction} in the \gls{os} is to create a on top of architecture specific code and abstract it in form of an internal \gls{api}.
This layer abstracts at least the \gls{cpu} and memory\cite[p.~5-7]{Arpaci-Dusseau2015}.
Higher-level complex management algorithms can be implemented hardware-independently on top of this \gls{api}, making it reusable across different architectures.
The \gls{os} then provides an \gls{api} through which \glspl{app} can request access to these virtualized resources.
Higher-level \glspl{proglang} allow the \gls{app} developers to develop and run different programs comfortably, concurrently and presumably safely on any hardware which is supported by the \gls{os}.
\Cref{fig:system-abstraction-layers} shows a top-down model of the abstractions layers in the system.

\begin{figure}[ht]
\begin{tikzpicture}
  [start chain=main going below, every on chain/.append style={align=center, text width=35ex, minimum height=7ex, fill=blue!20}, >={LaTeX[]}, node distance=7ex]
%  \node[on chain] {\gls{os} Abstraction Layers};
  \foreach \i/\itext [remember=\i as \iprior] in {
      1/Software Applications,
      2/System Libraries,
      3/OS API,
      4/OS {Core \& Drivers},
      5/{CPU, Memory, I/O, ...}
  }
  {
    \node (block \i) [on chain] {\itext};
    \ifnum\i>1
    \draw 
        (block \iprior.west) +(0ex,-1ex) coordinate (b\i) 
        edge["Synchronous API Calls" {left,text width=15ex,anchor=east, align=right}, bend right=90, ->] 
        (block \i.east -| b\i) +(0ex,+1ex);
    \draw 
        (block \iprior.east) +(0ex,-1ex) coordinate (k\i) 
        edge["Asynchronous Events" {right,text width=15ex,anchor=west, align=left}, bend left=90, <-]
        (block \i.east -| k\i) +(0ex,1ex);
    \fi
  }

  \draw[dashed,thick,red] 
      ($(block 2)!0.5!(block 3)$) ++(-20ex,0) edge["Userspace", "Kernelspace"'] ++(40ex,0);
  \draw[dashed,black,thick] 
      ($(block 4)!0.5!(block 5)$) ++(-20ex,0) edge["Software", "Hardware"'] ++(40ex,0);
\end{tikzpicture}
\caption{System Abstraction Layers}
% TODO: fill user/kernel space with colors
\label{fig:system-abstraction-layers}
\end{figure}

Virtualization has different technical implications for different resource types, depending on their nature and available count.
Programs that are executed need at least the \gls{cpu} and a certain amount of memory.

\subsection{Programming Languages} 
High-level \glspl{proglang} that support or include the \gls{os} \gls{api} allow the programmer to develop \glspl{app} without knowing \gls{os} or hardware internals.
This requires conventions to be defined which must then be adopted by all involved software components including and above the "OS API" layer in \cref{fig:system-abstraction-layers}.
For \gls{amd64}, the most popular calling convention is defined in the \citetitle{Matz2009}\cite{Matz2009}, which is supported by the major \gls{C} \glspl{compiler} GCC and CLANG, as well as by \gls{Rust}'s compiler.
\FloatBarrier

\subsection{Memory Virtualization: Paging}
Memory is a resource that is available in large quantities, made up from units reachable by single memory address.
The size of the smallest unit depends on the hardware and is ambiguously referred to as a \textit{memory word}, which is a fixed amount of Bytes depending on the architecture.
The whole memory can be divided into arbitrary groups of addressable units.
The \gls{os} supervises this structure, and programs must request memory from the \gls{os} to gain access to memory.
A method that combines virtualization and structuring is called paging and is explained in \cref{context::os-dev-concepts::hw-supported-mm}.

The goal of memory virtualization is to use \glspl{vaddr} in \glspl{program}, which can be dynamically mapped to physical memory addresses at system runtime.

\subsection{CPU Virtualization}
The \gls{cpu} is generally not explicitly requested by \glspl{program}, because any of the program's instructions implicitly requires the \gls{cpu} for being executed on the system at all.
Thus, the \gls{os} must chose at which time it executes which \gls{program} on the \gls{cpu}.

The goal of \gls{cpu} virtualization is to seamlessly share a single \gls{cpu} among all \glspl{program} and the \gls{os} itself, without changing any of the \glspl{program}.
How this can be achieved is explained in \cnameref{context::os-dev-concepts::preemptive-multitasking}.

\subsection{Program Execution}
\label{context::os-dev-concepts::virtualization::task}
As explained in this document it should be understood that a program consists of instructions that can be executed by the \gls{cpu}.
When the \gls{os} loads a program into memory and begins executing its instructions it is called a process.\cite[p.~25]{Arpaci-Dusseau2015}
A process can begin to exist before its execution, when the \gls{os} has internally created an entry for the process that at least contains a reference to the program and the arguments to be passed.
Processes that use the same program are not to be treated differently by the \gls{os} than any other process in terms of memory-safety, and must be prevented from mutual memory access.
These processes can differ in arguments that are passed to the program their, so that their runtime behavior can differ significantly.

\subsection{Terminology}
Terms that are often used to describe various forms of executed code are often used ambiguously in various documents, manuals, and websites. 
\Cref{tab:os-dev-concepts:task-terms} defines such terms and their relationships for the scope of this document.

\begin{table}
    \begin{tabularx}{\textwidth}{@{}lX@{}}
    \toprule
    \Gls{task}       & \glsentrydesc{task}      \\ 
    \Gls{program}    & \glsentrydesc{program}   \\ 
    \Gls{process}    & \glsentrydesc{process}   \\ 
    \Gls{thread}     & \glsentrydesc{thread}    \\ 
    \Gls{procedure}  & \glsentrydesc{procedure} \\
    \Gls{function}   & \glsentrydesc{function}  \\ 
    \Gls{job}        & \glsentrydesc{job}       \\ 
    \Gls{app}        & \glsentrydesc{app}       \\ 
    \bottomrule
\end{tabularx}
\caption{Definition of commonly used terms for executable code}
\label{tab:os-dev-concepts:task-terms}
\end{table}
\FloatBarrier


\subsubsection{Demo: Process $\neq$ Program}
A great example for demonstrating this difference is the program \textit{"\gls{bbox}. \glsentrydesc{bbox}"}.

Line 1 in \cref{shell::context::os-dev-concepts::program-process} shows a command that instructs the \gls{os} - \gls{LX} in this example - to execute the program \gls{bbox} three times, with different arguments each time.
The purpose is to demonstrate that the same program \gls{bbox} is instantiated thrice with completely different functionality each time, even existing simultaneously in the \gls{os}'s process list.

\begin{listing}[ht]
    \begin{minted}[escapeinside=??,linenos,autogobble,breaklines=true]{shell}
        $ busybox sh -c "busybox ps -Ao pid,args | busybox grep busybox" 
        9441 busybox sh -c busybox ps -Ao pid,args | busybox grep busybox
        9442 busybox ps -Ao pid,args
        9443 busybox grep busybox
    \end{minted}
    \caption{Multiple Process From Same Program}
    \label{shell::context::os-dev-concepts::program-process}
\end{listing}

It invokes \gls{bbox} with the \code{sh} (a shell utility) argument which in turn receives the \code{-c} (command execute" argument and another argument containing the expected command.
This command consists of subsequent calls to \gls{bbox} invoking its builtin \code{ps} (a utility to print the process list) and \code{grep} (a tool to find text) utilities.
The process list is received by passing \code{ps} as the argument to \gls{bbox} in the third execution.
Line 2 through 4 show the three \glspl{process} of the \gls{bbox} \gls{program} with different proccess ids and their respective arguments.

\section{Preemptive Multitasking Concepts}
\label{context::os-dev-concepts::preemptive-multitasking}
Multitasking is another vague term used in the \gls{os} jargon.
In this document, it is the \gls{os}'s capability of switching processes without terminating them, effectively keeping their runtime state in a place that persists while the task is not actively executed on the system's \gls{cpu}.
The \textit{preemptive} adds to multitasking the ability of switching tasks without relying on their cooperation, but instead be able to do this at any time the \gls{os} intends to do so.

Preemptive multitasking enables a form of \gls{cpu} virtualization, as a task is not aware of being preempted and resumed.  
As explained above, the \gls{cpu} resource doesn't have to be explicitly requested, as the request to execute a program implies a dependency on the \gls{cpu}..

The previously explained virtualization is the foundation for the \gls{os} to perform preemptive multitasking inconspicuously towards the \glspl{app}.
This means that when a task is preempted and continued later, it observes no side-effects other than an elapse of time.
Preemptive multitasking needs not be considered during development of single-threaded \gls{app}.

\subsection{Concurrent Resource Usage}
Switching tasks has different technical implications for different resources types, depending on their nature and quantity.

A single \gls{cpu} system can not execute more than one program at the same time, as it runs instructions through the \gls{cpu} one-by-one, implicitly holding the program state in form of the \gls{cpu} registers, which are preserved in between the instructions, and preserved between preemptive task switches.

While it doesn't make sense for any instruction to request the \gls{cpu} per-se, there are valid use-cases for programs to request a specific amount of \gls{cpu} bandwidth within a specific amount of time to guarantee a certain amount of computing speed.
Other use-cases emerge when concurrent programs access the exact same resources and are intermixed by the \gls{os}, creating non-sequential resource usage pattern which may put the resource in an inconsistent state and lead to unexpected results for the application.
Within this document these technicalities are considered part of the application semantics and shouldn't affect the \gls{os} development.

Hence for this document it is sufficient to recognize the \gls{os}'s responsibility of cleanly switching the program in execution periodically.

In contrast to the \gls{cpu}, the main memory resource is available in limited but huge quantities.
Replacing the content of the memory is not necessary on a preemptive task switch, as long as the memory is not exhausted. 
This has the effect that tasks which are currently not in execution on the \gls{cpu} still own a region of main memory.

The \gls{os} must ensure that switching tasks is done properly for all resources to prevent interference and unintended behavior.
To ensure memory safety in this scenario, all data in the memory must be protected from unintended access, according to the definition of memory safety in \cref{context::introduction::memory-safety::def}.

\subsection{Context Switching}
The context switch is the core functionality of the multitasking as it effectively switches to a different process, possibly by preempting the one that is currently running.
When the \gls{os} preempts a process, it needs to store and preserve the current process's context.
The context consists of all volatile resources that can possibly be overwritten by another process.
This is at minimum a set of \gls{cpu} registers depending on the specific architecture.

The \gls{os} stores the preempted context in a well-known and protected memory location, so that it can be restored when this process is resumed.

In preemptive multitasking, context switches are not considered voluntary, but rather by force.
This works by using the \gls{cpu}'s interrupt mechanism which has the ability to jump to an \gls{os} function in the event of an interrupt.
Interrupts for this use-case are usually triggered by programmed timer interrupts, occurring continuously and regularly.
The interrupt is signaled to the \gls{os} by \gls{cpu}, so the lowest level of the process switching mechanism in the \gls{os} takes place in a hardware specific interrupt handler.

Safety could be increased if the \gls{compiler} or in a more general sense the \gls{proglang} could assist in architecture specific code.
More details on this mechanism is given in \cref{context::os-dev-concepts::preemptive-multitasking-amd64}, but first it is necessary to understand the involved memory management mechanisms which are explained in \cref{context::os-dev-concepts::hw-supported-mm,context::os-dev-concepts::stackheap,context::os-dev-concepts::sf-handling-amd64,context::os-dev-concepts::stackheap-combined}.

\section{Hardware-supported Memory-Paging}
\label{context::os-dev-concepts::hw-supported-mm}
To improve the efficiency and safety of memory management, developers of hardware and software have been collaborating to offload virtual memory address lookup and caching from the \gls{os} software to the hardware, the \gls{cpu}'s \gls{mmu} to be specific.
A hardware-implementation of the lookup algorithm is fast, and allows rudimentary memory permission runtime-checks to protect pages by leveraging \gls{cpu}'s security rings\cite[p.~117,~p.~145]{AMD64Vol2}.

Activating the 64-Bit long mode on \gls{amd64} makes the system rely primarily on paging memory management, thus the technique of memory segmentation can be neglected in this context.
This section provides information about hardware-supported memory paging and protection techniques.

\subsection{Virtual Address Translation and Paging}
Paging with \glspl{vaddr} is one method of virtualizing memory and in this way transparently share the system's memory among running tasks and the \gls{os} itself, presumably in a safe way. 
Even when using a language that supports direct memory addressing, \gls{app} developers don't have to consider paging and address translation in the logic of their programs, because all addresses in their program are virtual and are translated at runtime by the \gls{mmu}.
The translation itself is performed by the \gls{mmu} according to a map that is called page table, which is a structure maintained by the \gls{os} in the main memory.
This memory structure can be stored anywhere in memory, and the address is handed to the \gls{mmu} via a specific \gls{cpu} register, \textit{CR3} on \gls{amd64}.
The \gls{os} can maintain multiple page table structures, and can create different \gls{vaddr} spaces by changing the \gls{mmu}'s page-table pointer register.

\subsubsection{Translation Caching}
The hardware caches the translation results for subsequent lookups in the \gls{tlb} \cite[p.~142-143]{AMD64Vol2}.
This greatly improves the speed for repeated access to the same \glspl{vaddr}, but is certainly dangerous for memory-safety.
Controlling the validity of these cache entries is the responsibility of the \gls{os} .
This is critical for memory-safety, as the cached virtual to physical address lookup results are different for each address space and shouldn't leak into other address spaces.

If any lookup yields a cached result which originates from a different \gls{vaddr} space, the physical address is likely to belong to a memory region to which the current task shouldn't have access to.
What makes it more difficult to manage is that there are exceptions to this, e.g. when memory is intentionally shared between two processes or threads, which must be set up by the \gls{os} according to the processes requests.

\subsubsection{Pages: Chunks of Smallest Addressable Unit}
To avoid the need for storing a translation mapping for every single \gls{vaddr} , mappings are grouped into equisized regions, called \textit{page}s.
This works by encoding a page-offset in the \gls{vaddr} , together with page's index in the page table.
The offset size depends on the chosen page-size, and can be calculated with the following formula, given page-size $p$, a power of two given in Byte:
\begin{equation}
    f(p) = log_2(p)
\end{equation}
For example, the \gls{amd64} default page-size of 4 KiB has a 12-bit offset, which theoretically leaves the other $64-12 = 52$ bits of the \gls{vaddr} for page-table indexing.
On \gls{amd64} there's an architectural limit of 48 bits for the \gls{vaddr} , thus the address constellation is different than explained here. 
Details on this concrete implementation follows in \cnameref{context::os-dev-concepts::hw-supported-mm::multilevel-paging-amd64}.

\subsection{Page-Faults}
\label{context::os-dev-concepts::hw-supported-mm::page-fault}
To improve the efficiency and safety of memory management, developers of hardware and software have been collaborating to offload virtual memory address lookup and caching from the \gls{os} software to the hardware, the \gls{cpu}'s \gls{mmu} to be specific.
The page-fault is a hardware-triggered, memory-safety critical event that must be handled by the \gls{os}.
It is triggered by the \gls{cpu}'s \gls{mmu} during the \gls{vaddr} lookup algorithm, when an instruction uses a \gls{vaddr} for which the target page is not available.
This happens for example if the indexed page is not present in main memory or has not been allocated at all.
It also happens when an instruction violates a page protection, of which four exist and can be configured by the \gls{os} on \gls{amd64} \cite[p.~145-148]{AMD64Vol2}:
\begin{itemize}
    \item (1) Protect supervisor pages from user access
    \item (2) Prevent writes into read-only pages
    \item Prevent the \gls{cpu} from executing (3) non-executable pages and (4) user pages
\end{itemize}

The \gls{os} must implement the page-fault handler to deal with it accordingly.
For example, the case of a non-existing mapping requires to find and allocate free physical memory and map it to the page by modifying the \gls{vaddr} e's page-table entry.
Or in case of protection violation it would simply indicate denied access.

\subsection{Swapping}
The finite primary memory can only hold a finite number of virtual pages, and the \gls{os} is responsible for having the required pages present.
Besides the pages that contain the page-table itself, the pages that aren't required by the current instruction might be moved to secondary memory.
Swapping pages in and out of primary memory is risky as it requires to transfer large amounts of raw memory content, but these safety analyzes exceed the scope of this study.

\subsection{Multi-Level Paging Concept}
\label{context::os-dev-concepts::hw-supported-mm::multilevel-paging-concept}
If only one page-table per \gls{vaddr} space was used that consists of $2^{52}$ page-table entries, which must at minimum store the physical address, it would require $\frac{52 * 2^{52} [Bit]}{8*1024^4 [Bit/Byte]} = 26624$ TiB of memory for each \gls{vaddr} space.
Even if only a handful of additional pages were allocated and mapped, the \gls{os} would still have to allocate this huge page-table.
This vast consumption of main memory is impractical and impossible for average systems, which rarely surpass 100 GiB of main memory.

Therefore most systems use a hierarchy of page tables.
Using a hierarchical translation structure allows to save significant amounts of memory, as not every page-table of every level in the address space has to be allocated and present in main memory.

% TODO picture this
\begin{figure}
\centering
\includegraphics[width=\textwidth]{gfx/TODO-nlevel-paging}
\begin{tikzpicture}
    \def\x{9ex}
    % memory cells
    \path[draw,font=\small]
     % cells
    (0,0)
        rectangle ++(\x, 1) 
        rectangle ++(\x,-1)
        rectangle ++(\x, 1) 
        rectangle ++(\x,-1)
        rectangle ++(\x, 1) 
    % cell text
    (0.5*\x,0.5)
        node(text-a){idx$_n$} ++(\x,0)
        node(text-b){idx$_{n-1}$} ++(\x,0)
        node(text-c){...} ++(\x,0)
        node(text-d){idx$_1$} ++(\x,0)
        node(text-e){offset$_{page}$}++(\x,0)
    % bit numbers
    (0,1) 
        node[anchor=south]{63}
    (5*\x,1) 
        node[anchor=south]{0}
        ;
    % braces
    \foreach \y in {1,...,5} {
        \pgfmathparse{\y-1}
        \draw[decorate,decoration={brace,mirror}] 
            ($(\pgfmathresult*\x,-1ex)!0.1!(\y*\x,-1ex)$) -- node[shape=coordinate](brace-\y){}
            ($(\pgfmathresult*\x,-1ex)!0.9!(\y*\x,-1ex)$);
    }

    \draw
     % cells
    (0*\x,-1*\x)                  
        rectangle ++( \x, 0.5*-1) 
        rectangle ++(-\x,-1)     
        rectangle ++( \x, 0.5*-1) 
    (1*\x,-3*\x)
        rectangle ++( \x, 0.5*-1)
        rectangle ++(-\x,-1)
        rectangle ++( \x, 0.5*-1) 
    (2*\x,-5*\x)
        rectangle ++( \x, 0.5*-1)
        rectangle ++(-\x,-1)
        rectangle ++( \x, 0.5*-1) 
        ;

    % cell text
%    (0.5*\x,0.5)
%        node(text-a){idx$_n$} ++(\x,0)
%        node(text-b){idx$_{n-1}$} ++(\x,0)
%        node(text-c){...} ++(\x,0)
%        node(text-d){idx$_1$} ++(\x,0)
%        node(text-e){offset$_{page}$}++(\x,0)
%    % bit numbers
%    (0,1) 
%        node[anchor=south]{63}
%    (5*\x,1) 
%        node[anchor=south]{0}
%        ;

%        \def\y{1}
%        \pgfmathparse{\y-1}
%        \draw[decorate,decoration={brace,mirror}] 
%            ($(\pgfmathresult,-1ex)!0.1!(\y*\x,-1ex)$) -- node[shape=coordinate](brace-\y){}
%            ($(\pgfmathresult,-1ex)!0.9!(\y*\x,-1ex)$);
%        \def\y{2}
%        \pgfmathparse{\y-1}
%        \draw[decorate,decoration={brace,mirror}] 
%            ($(\pgfmathresult*\x,-1ex)!0.1!(\y*\x,-1ex)$) -- node[shape=coordinate](brace-\y){}
%            ($(\pgfmathresult*\x,-1ex)!0.9!(\y*\x,-1ex)$);
%        \draw[decorate,decoration={brace,mirror}] 
%            ($(1*\x,-1ex)!0.1!(\x*2,-1ex)$) -- node[shape=coordinate](brace-2){} 
%            ($(1*\x,-1ex)!0.9!(\x*2,-1ex)$);

\end{tikzpicture}
\caption{Hierarchical Virtual Paging}
\label{fig:paging-hierarchy-abstract}
\end{figure}
\FloatBarrier

\subsection{Multi-Level Paging on AMD64}
\label{context::os-dev-concepts::hw-supported-mm::multilevel-paging-amd64}
On \gls{amd64} "a four-level page-translation data structure is provided to allow long-mode operating systems to translate a 64-Bit virtual-address space into a 52-Bit physical-address space."\cite[p.~18]{AMD64Vol2}.
This allows the system to only hold the \textit{PML4} table, the which is currently referenced by the \textit{Page Map Base Register (CR3)}, available in main memory.

\Cref{fig:virtual-addr-transl} shows the 64-Bit \gls{vaddr} composition on \gls{amd64}, which uses four-levels of page tables.
Counter intuitively the page-tables are not called level-\textit{n}-page-table, but the levels received distinct names in \citetitle{AMD64Vol2}.
The most-significant Bits labelled as \textit{Sign Extend} are not used for addressing purposes, but must adhere the canonical address form and simply repeat the value of the most-significant implemented Bit \cite[p.~130]{AMD64Vol2}.
The least significant Bits represent the offset within the physical page.
The four groups in between are used to index the page-table at their respective level.

\begin{figure}
\centering
\includegraphics[width=\textwidth]{gfx/Virtual-to-Physical-Address-Translation-Long-Mode.png}
\caption{Virtual to Physical Address in Long Mode\cite{AMD64Vol2}}
\label{fig:virtual-addr-transl}
\end{figure}
\subsubsection{Translation Scheme 4 KiB and 2 MiB Pages}
The \gls{amd64} architecture allows configuring the page-size, two of which will be introduced in this section.
\cref{tab:page-transl-vaddr-composition} displays the \gls{vaddr} composition for the 4KiB and 2MiB page-size modes on \gls{amd64}.
The direction from top to bottom in the table corresponds to most significant to least significant - left to right - in the \gls{vaddr} .
The \textit{sign extension} Bits cannot be used for actual information but act as a reservation for future architectural changes.

\begin{table}
    \begin{tabular}{l | c | c}
        Description & Bits in 4 KiB Pages & Bits in 2 MiB Pages \\
        \hline
        Sign Extend & 12 & 12 \\
        Page-Map-Level-4 Offeset & 9 & 9 \\
        Page-Directory-Pointer Offeset & 9 & 9 \\
        Page-Directory Offeset & 9 & 9 \\
        Page-Table Offeset & 9 & - \\
        Physical Page Offset & 9 & 21 \\
    \end{tabular}
    \caption{Paging on \gls{amd64}: Virtual Address Composition 4KiB/2MiB pagesizes}
    \label{tab:page-transl-vaddr-composition}
\end{table}

\begin{figure}
\centering
\includegraphics[width=\textwidth]{gfx/amd64-4kb-page-translation-long-mode}
\caption{4-Kbyte Page Translation—Long Mode\cite{AMD64Vol2}}
\label{fig:4kb-page-transl}
\end{figure}

\cref{fig:4kb-page-transl} shows the detailed \gls{vaddr} composition for 4 KiB pages, using four levels of page-tables.
It uses four sets of 9-Bit indices in the \gls{vaddr} , one per hierarchy level, followed by the 9 Bit page-internal offset.

An alternative approach is displayed in \cref{fig:2mb-page-transl}, using 2 MiB sized pages.
It uses three sets of 9-Bit indices for the page-tables, and a 21-Bit page-internal offset.
Increasing the page-size improves speed and memory-usage and decreases the granularity.
In this specific example the hierarchy is reduced by one level of page-tables.
This reduces the amount of storage required for the page-tables in overall and causes the lookup algorithm to finish faster.

\begin{figure}
\centering
\includegraphics[width=\textwidth]{gfx/amd64-2mb-page-translation-long-mode}
\caption{2-Mbyte Page Translation—Long Mode\cite{AMD64Vol2}}
\label{fig:2mb-page-transl}
\end{figure}

The other supported page sizes, 4 MiB and 1 GiB, as well as intermixing page sizes through the different levels don't add new insight into the mechanism and don't need to be detailed here.

\section{Stack And Heap: Basic Concepts}
\label{context::os-dev-concepts::stackheap}
In \gls{proglang} and \gls{os} literature, the terms \gls{stack} and \gls{heap} are ubiquitous.
A research for their original definition wasn't conclusive, indicating that they are to be taken as concepts rather than absolutely defined methods.
They might be implemented and used differently on various architectures, \glspl{proglang} and \glspl{os}.

The hardware manuals \citetitle{AMD64Vol1} and \citetitle{AMD64Vol2} have no mention of the word \textit{heap}, but use \textit{stack} hundreds of times, indicating that only the \gls{stack} concept is implemented in hardware.
It is comparable to computers that had physical rewindable memory tapes, which is conceptually similar to the \gls{stack} implementations in today's memory management.

Likely the first mention of the term \gls{heap} with regard to memory management is found in \citetitle{Mailloux1969}.
It was introduced featuring "dynamic storage allocation", further described to have "the ability to create and manipulate objects whose lifetimes are not so restricted. This ability implies the use of an additional area of storage, the “heap”, in which garbage-collection techniques must be used."\cite[p.~8]{Mailloux1969}
Since then, the \gls{heap} has been implemented in all major \glspl{proglang}.

This sections explains both concepts with the tendency towards the \gls{amd64} architecture, the \gls{C} and \gls{Rust} \glspl{proglang} and their usage for bare-metal \gls{os} and \gls{LX} \gls{app} development.

\subsection{Stack: Hardware-Backed Abstract Type}
\label{context::os-dev-concepts::stackheap::stack}
In summary, the \gls{stack} is a memory model for structuring a contiguous region of memory.
It grows by adding new data entries on top of each other.
According to the \gls{stack} analogy, only the topmost element can be accessed and removed, thus it behaves like a Last-In-First-Out data structure.

The \gls{amd64} manuals conjunctionally describe how the \gls{stack} is used and influenced by various instructions on this architecture.
Here it grows from numerically higher to numerically lower addresses, whereas the numerically highest address is called the stack bottom, and the current numerically lowest address is the stack top.
In 64-Bit long mode on \gls{amd64}, the \gls{cpu} doesn't consider the stack to be sized or explicitly bounded, which is highly safety critical.

The \gls{stack} is typically allocated per process or thread and is used to store each procedure's \gls{sf}.
Each \gls{sf} is automatically cleaned up or simply forgotten once the procedure has completed.
When multiple procedure calls are nested, each preceding \gls{sf} remains on the stack in front of the next one.
\cref{fig:stack-with-two-frames}\footnote{By R. S. Shaw - Own work, Public Domain, \url{https://commons.wikimedia.org/w/index.php?curid=1956587}} displays a \gls{stack} that contains two \glspl{sf} from different procedures.
In this picture, the \gls{stack} grows upwards.

\begin{figure}[ht]
\centering
%\begin{wrapfigure}{R}{0.5\textwidth}
    \includegraphics[width=0.49\textwidth]{gfx/TODO-Callstacklayout}
    \caption{An upwards growing stack with two stack-frames}
    \label{fig:stack-with-two-frames}
%\end{wrapfigure}
\end{figure}
% TODO: draw stack with multiple stack-frames
\FloatBarrier

\subsubsection{Safety Concerns}
To achieve memory-safe \gls{stack} management in the \gls{os}, each procedure must only be able to access its own particular \gls{stack} and possible references via its arguments.
This is not the case in a regular \gls{C} program, where the called procedure is able to modify the previous \gls{sf}, which is demonstrated in \cnameref{rnd::weakness-mitig-prev::stack-protection}.
Additionally, the \glspl{stack} must be prevented from growing into other memory zones like the \gls{heap}.
Since \Gls{stack} management is memory-safety critical for \gls{os} developers when implementing memory management for multitasking within the \gls{os}; This is one of the main subjects in \cref{rnd::existing-os-dev-with-rust,rnd::imezzos-preemptive-multitasking}.

\subsection{Heap: Organized Chaos}
\label{context::os-dev-concepts::stackheap::heap}
\Gls{heap} is an ambiguous term that names two computer related models.
A data structure in theoretical computer science, and a memory model in system resource management.
This document refers to the latter.

The \gls{heap} is managed by the \gls{os} to keep track of allocated memory on behalf of all \glspl{app} on the system.
The algorithms within the \gls{os} to manage the \gls{heap} can be arbitrary complex, and the choice is based on the trade-offs between complexity, efficiency and speed. 
Safety is explicitly omitted from the trade-offs list, as it should never be traded against anything else.

\Gls{app} developers make use of dynamic \gls{heap} allocation requests via the \gls{os}'s \gls{api} if the memory usage for their program is not predictable at the time of development.
The requests will be processed by the \gls{os} and granted or denied according to ratio of requested memory and available system memory.
The \glspl{app} is responsible for returning no-longer required memory to the \gls{os}, which makes it available to other \glspl{app} in the system.

Memory which is not cleaned up by properly is blocked until the \gls{app} is terminated.
If \gls{heap} allocations within the \gls{os} are not cleaned up properly, the allocated memory is lost until the whole system is restarted.
Between properly cleaning up and loosing memory allocations is a whole range of possible memory-safety issues, which are explained in \cref{context::introduction::memory-safety-violation-in-sw}.

\section{Stack Frame Handling on AMD64}
\label{context::os-dev-concepts::sf-handling-amd64}
The usage of the \gls{stack} is tightly coupled with control flow instructions in conjunction with two registers, the Stack-Frame Base Pointer (RBP) and the Stack Pointer (RSP).
The instructions that use these registers and explicitly or implicitly work with the stack\cite[p.~83]{AMD64Vol1} can be grouped into the following categories.
Together they can be used to perform \gls{stack} based procedure calls, as demonstrated in the following \cref{context::os-dev-concepts::sf-handling-amd64::procedure-call-example}.

\subsection{Direct Stack Data Management Instructions}
\code{push} a takes value operand which is to be pushed onto the stack.
The address in RSP moves towards numerically lower addresses with every \code{push} instruction, which stores a new data entry on top.
The order is to first change the RSP and then copy the value at its new address.

\code{pop} takes a storage reference operand - \gls{cpu} register or memory address.
It works in the opposite direction to \code{push}.
First, consuming the top-most data entry and storing it on the operand location, then moving the RSP address towards the numerically higher RBP address.

When RBP and RSP point to the same address, the stack is considered empty.

\subsection{Procedure Calls Instructions}
The \code{call} and \code{ret} instructions control the instruction flow by calling another instruction procedure\footnote{loosely synonymous with function}.


The \code{call} instruction takes the address of the instruction that is to be called.
Before jumping to the instruction at the given address, it \code{push}es the current RIP (instruction pointer) register onto the \gls{stack}.

\code{ret} takes no operand, but instead \code{pop}s the \gls{stack}'s top entry.
The consumed value is used as a jump address.

As \code{push} and \code{pop} use the RSP register, the called procedure is responsible to finish with the RSP at the same position as when it was entered.
For example, \code{push}ing some value onto the stack before the end of the function would cause the \code{ret} to jump to that address instead of returning to the caller.

\subsection{Stack Frame Management Instructions}
When a procedure is called, the stack is set up with the \gls{sf}, the four components listed in \cref{lst:amd64-stack-frame-components}.
\cite[p.~48]{AMD64Vol1}:

\begin{listing}[h]
\begin{enumerate}
    \item{%
        Parameters passed to the called procedure (created by the calling procedure). \\
        \textit{Only if parameters don't fit the \gls{cpu} registers}
    }
    \item{%
        Return address (created by the \code{call} instruction). \\
        \textit{Always used by \code{call}}
    }
    \item{%
        Array of stack-frame pointers (pointers to stack frames of procedures with smaller nesting-level depth) which are used to access the local variables of such procedures. \\
        \textit{Depends on support and implementation of nested functions in the \gls{compiler}}
    }
    \item{%
        Local variables used by the called procedure. \\
        \textit{This includes the variables passed via \gls{cpu} registers}
    }
\end{enumerate}
\caption{\glsentrytext{amd64} Stack-Frame Components}
\label{lst:amd64-stack-frame-components}
\end{listing}

The \gls{amd64} manual also lists \code{enter} and \code{leave} as instructions to \textit{"provide support for procedure calls, and are mainly used in high-level languages."}\cite[p.~48]{AMD64Vol1}.
The latter claim could not be verified by inspecting binaries produced by the \gls{C} and \gls{Rust} \glspl{compiler}.

Instead, these \glspl{compiler} generate a sequence of \code{push}, \code{mov} and \code{sub} instructions to manage theset up the \gls{stack}.
There are instructions before and after the procedure's logic, taking care of the technicalities of \gls{stack} management.
These instruction groups within the called procedure are called prologue and epilogue.

\subsection{Full Procedure Call Example}
\label{context::os-dev-concepts::sf-handling-amd64::procedure-call-example}
This section combines the separate categories into one complete example that shows how the \gls{stack} is used by various \gls{cpu} instructions to perform procedure calls.
The following code samples are extracted from a disassembled binary which was originally created using \gls{Rust}.
The Assembler that's shown uses Intel Mnemonic, which generally operates from right to left.
For example, \code{mov a, b} copies b to a.

\cref{code::context::examples::func-callee-rust} shows the \gls{Rust} source code of the function \textit{sum}.

% \subsubsection{Top-Level Page Table Self-Reference}
% \subsubsection{Caching Lookups}
% \subsubsection{Full Example}
% * http://taptipalit.blogspot.de/2013/10/theory-recursive-mapping-page.html
% * https://www.coresecurity.com/blog/getting-physical-extreme-abuse-of-intel-based-paging-systems-part-2-windows

\begin{listing}[htb]
    \tikzset{/minted/basename=callee-rust}
    \begin{minted}[autogobble,linenos,breaklines=true]{rust}
    TODO
    \end{minted}
    \caption{Procedure Call Example: Callee in Rust}
    \label{code::context::examples::func-callee-rust}
\end{listing}

\begin{listing}[htb]
    \tikzset{/minted/basename=callee-rust}
    \begin{minted}[autogobble,linenos,breaklines=true]{nasm}
    TODO
    \end{minted}
    \caption{Procedure Call Example: Callee in Assembly}
    \label{code::context::examples::func-callee-assembly}
\end{listing}

\Cref{code::context::examples::func-caller-asm} shows a snippet of the calling function. 
It stores the arguments within the registers according to the calling convention.
The caller doesn't alter the stack-frame pointer (RBP) or the stack pointer (RSP) registers before call, hence the called function must restore these if it alters them.

\begin{listing}
    \begin{minted}[escapeinside=??,highlightlines={},autogobble,linenos,breaklines=true]{nasm}
    TODO
    \end{minted}
    \caption{Procedure Call Example: Caller Assembly}
    \label{code::context::examples::func-caller-asm}
\end{listing}

\begin{listing}
    \begin{minted}[escapeinside=??,highlightlines={},autogobble,linenos,breaklines=true]{rust}
    \end{minted}
    TODO
    \caption{Procedure Call Example: Caller in Rust}
    \label{code::context::examples::func-caller-rust}
\end{listing}

% \balloon{comment}{

% RDI, RSI, RDX, RCX, R8, R9, XMM0–7

\begin{table}[ht!]
    \centering
    \begin{tabular}{ r | >{\columncolor{YellowGreen}}c | l }
        \multicolumn{1}{r}{RBP offset} & \multicolumn{1}{c}{Content} & \\
        $\uparrow$ & \cellcolor{white} & \\
        & \cellcolor{white} \dots \textit{beyond current stack} \dots & \\
        \hhline{~-~} 
        0 & \textit{Previous RSP} & $\leftarrow$ RBP \\
        \hhline{~-~} 
        \vdots & \dots~~\textit{local variables}~~\dots & \\
        \hhline{~-~} 
        -0x30 & 3rd arg & \\
        \hhline{~|-|~}
        -0x38 & 2nd arg & \\
        \hhline{~-~}
        -0x40 & 1st arg & \\
        \hhline{~-~} 
        \vdots & \dots~~\textit{local variables}~~\dots & \\
        \hhline{~-~} 
        -0x60 & rdi & \\
        \hhline{~-~} 
        & \dots~~\textit{local variables}~~\dots & \\
        \hhline{~-~} 
        $RBP-RSP$ & \textit{unknown} & $\leftarrow$ RSP \\
        \hhline{~-~} 
        & \cellcolor{white} & \\
        $\downarrow$ & \cellcolor{white} & \\
    \end{tabular}
\end{table}

\cref{code::context::examples::func-prologue} shows \textit{sum}'s prologue.
The corresponding epilogue is displayed in \cref{code::context::examples::func-epilogue}. 
The comments explain the code line by line, please read them to understand what exactly happens at each instruction.

\begin{listing}[ht!]
\begin{minted}[escapeinside=??,linenos=false,breaklines=true]{nasm}
$7490: push   rbp       ; save the stack-frame pointer on the stack
$7491: mov    rbp,rsp   ; set the stack-frame base pointer from the stack pointer
$7494: sub    rsp,0x50  ; allocate 0x50 Bytes for arguments and local variables
$7498: mov    QWORD PTR [rbp-0x30],rdi ; copy 1st arg onto stack 
$749c: mov    QWORD PTR [rbp-0x28],rsi ; copy 2nd arg onto stack 
$74a0: mov    QWORD PTR [rbp-0x20],rdx ; copy 3rd arg onto stack  
\end{minted}
\caption{Function Prologue with three Arguments}
\label{code::context::examples::func-prologue}
\end{listing}

\begin{listing}[ht!]
\begin{minted}[linenos=true,breaklines=true]{nasm}
$74ee: mov    rax,QWORD PTR [rbp-0x48] ; store return value in RAX
$74f2: add    rsp,0x50                 ; set stack pointer to where stack-frame pointer was stored
$74f6: pop    rbp                      ; restore the stack-frame pointer
$74f7: ret                             ; return to the caller, following the address on the stack
\end{minted}
\caption{Function Epilogue}
\label{code::context::examples::func-epilogue}
\end{listing}

\cref{fig:proc-call-example-mem} displays 

\begin{figure}
\centering
\includegraphics[width=0.95\textwidth]{gfx/call-procedure-memory-content.png}
\caption{Memory Layout Throughout The Procedure Call Steps}
\label{fig:proc-call-example-mem}
\end{figure}
\FloatBarrier

\section{Stack And Heap: Combined Usage}
\label{context::os-dev-concepts::stackheap-combined}
\Glspl{program} combine the usage of \gls{stack} and \gls{heap} use them for different purposes.

Depending on the \gls{proglang}'s \gls{compiler} and the target system, the responsibility of writing the memory management code falls either on the developer, the \gls{compiler}, or both. 
\Cref{tab:stack-heap-usage-responsbility} describes the usage-cases and responsibilities for \gls{stack} and \gls{heap} with regard to user-space and \gls{os} development.
 
\begin{table}[ht!]
    \begin{tabularx}{0.99\textwidth}{XX}
        \toprule
        \multicolumn{2}{c}{Stack and Heap Usage Overview} \\
        \toprule
        \multicolumn{1}{c}{\Gls{stack}} & \multicolumn{1}{c}{Responsibility} \\
        \hhline{--}
            \Gls{sf} (return address, frame pointer, see \cref{context::os-dev-concepts::stackheap::stack}).

            Procedure-local fixed-sized variables (primitive types, custom fixed-size structures, references, fixed-length arrays, etc.)

            Procedure arguments (allocated as and copied to local variables)
        &
            In user space development the \gls{stack} management code is fully generated \tikzmarkcircle{1} by the \gls{compiler} in \gls{C} and \gls{Rust}, using a standardized calling convention for the platform.
            The \gls{os} must implement support for any standardized calling conventions used by the \gls{compiler} for user-space \glspl{program}.
        \\
        \bottomrule
        \multicolumn{1}{c}{\Gls{heap}} & \multicolumn{1}{c}{Responsibility} \\
        \toprule
            Dynamically sized data structures like linked-lists, extendable arrays, queues, trees

            Can be used for unstructured arbitrary data.
        & 
            Manual calls for allocation and freeing memory are written by the programmer in \gls{C} using the \gls{os} \gls{api}.
            \gls{Rust} has support of generating these calls on behalf of the programmer on supported platforms via its standard library \tikzmarkcircle{2}.
        \\
        \bottomrule
    \end{tabularx}
    \begin{tabularx}{\textwidth}{l@{ }X}
        \tikzmarkcircle{1} & 
        The \gls{app} programmer is in control of the source code that triggers the auto generated content.
        E.g. defining a function with local variables in the code will yield generated instructions to handle the \gls{sf}. \\
        \tikzmarkcircle{2} &
        The origin of the auto generated content remains the source code, written by the programmer.
        Therefore it is the choice of the programmer whether to place the variables on the \gls{stack} or on the \gls{heap}.
    \end{tabularx}
\caption{Stack and Heap Usage and Responsibility}
\label{tab:stack-heap-usage-responsbility}
\end{table}
\FloatBarrier

\subsection{Arrangement}
\label{context::os-dev-concepts::stackheap-combined::arrangement}
Both zones must be organized separately and arrange within the \gls{vaddr} space which is assigned to process or thread.
\Cref{fig:heap-malloc-stack-example-program} shows a \gls{C} \gls{program} and a simplified model of the hypothetical address space that would result on execution.
In this example, the \gls{stack} and \gls{heap} are placed on opposite sides of the \gls{vaddr} space, and will grow towards each other.

\begin{figure}[ht!]
    \centering
    \includegraphics[width=0.6\textwidth]{gfx/TODO-heap-stack-example-program}
    %\caption{Stack/Heap Arrangement And Dynamic Allocation \footnote{Prof. Jennifer Rexford, \url{http://slideplayer.com/slide/3288060/}}}
    \label{fig:heap-malloc-stack-example-program}
    % TODO: redraw
\end{figure}
\FloatBarrier

The entries above \textit{"Heap"} are the different parts of the \gls{compiler} output for this program, and are loaded by the \gls{os} before the execution.
The colors in the code correspond to the entries in the memory model.
Text, which corresponds to the red box, contains the program instructions.
RoData is read-only memory content, in this case the string literal \textit{"string"}.
BSS contains the variable \textit{iSize}.
Lastly the \gls{stack} holds the pointer variable \textit{p}, which will reference the result of the \textit{malloc(iSize)} memory allocation.

\subsection{Safety Concerns}
\label{context::os-dev-concepts::stackheap-combined::safety-concerns}
Even though \gls{vaddr} spaces are huge on \gls{amd64}, there is a slight chance that the \gls{stack} and \gls{heap} will interfere. 
This could be due to direct collision, or more subtly by not detecting invalid mutual references.


\subsection{Programming Language Support}
In many \glspl{proglang} that are commonly used for \gls{app} development, the code for allocation and cleanup of \gls{heap} memory is generated by the \gls{compiler} on behalf of the programmer.
Such languages rely on the \gls{os} memory management \gls{api} and are thus not suited for developing the \gls{os} itself.
Visa-versa, languages which are suited for \gls{os} development usually don't generate \gls{heap} management code and therefore don't ensure memory-safety on the \gls{heap}.

\gls{Rust} might be an exception to this which has to be confirmed or denied by the end of this work.

\section{Preemptive Multitasking on \glsentrytext{amd64}}
\label{context::os-dev-concepts::preemptive-multitasking-amd64}
On \gls{amd64}, the \gls{cpu}'s interrupt mechanism does not switch the full content of the context, but only handles the registers that are necessary to successfully jump to the interrupt function: RFLAGS, RSP, RBP, RIP\footnote{Segment registers are neglected}.

\subsection{The Process Context}
The content of a process's context on \gls{amd64} is given in \cref{tab:task-minimum-context-registers}.
All these registers need to be stored and restored by the \gls{os}'s interrupt handler for process preemption.

\begin{table}
    \begin{tabularx}{\textwidth}{| c | X | X |}
    \hline
        \textbf{descriptive name} & 
        \textbf{register names on amd64} &
        \textbf{description} \\ 
    \hline
    the instruction pointer register & RIP & address of the next instruction to be fetched \\
    \hline
    the stack pointer register & RSP & address of current position in stack \\
    \hline
    the flags register & RFLAGS & various attributes, e.g. the interrupt flag \\
    \hline
    all general-purpose registers & RAX, RBX, RCX, RDX, RDI, RSI, RBP, RSP, R8–R15  & arbitrary data \\
    \hline
    \end{tabularx}
    \caption{Minimum Context Registers on amd64\cite[p.~28]{AMD64Vol2}}
    \label{tab:task-minimum-context-registers}
\end{table}

\subsection{Storing The Context On The Stack}
In this scenario, the context is stored on the \gls{stack} of the function that is interrupted.
\Cref{fig:amd64-long-mode-interrupt-stac} pictures the \gls{stack} layout on interrupt entry.
In order to leverage an interrupt for a context switch, the interrupt function needs to replace these values on the \gls{stack} with values for the new context.
CS (Code-Segment) and SS (Stack-Segment) have no effect in \gls{amd64} 64-Bit mode\cite[p.~20]{AMD64Vol1} and can remain unchanged.
The \gls{os} developer needs to know the exact address where on the \gls{stack} this data structure has been pushed by the \gls{cpu}, and must then manipulate these addresses directly.
This type of manipulation is inherently dangerous and can not be easily checked by the \gls{compiler}.
The function that handles the interrupt must then use the instruction \textit{iretq}\cite[p.~252]{AMD64Vol2}, to make the \gls{cpu} restore the partial context from the \gls{stack} and continue to function pointed to by the RIP.

\begin{figure}
\centering
\includegraphics[width=0.8\textwidth]{gfx/amd64-long-mode-stack-after-interrupt.png}
\caption{Long-Mode Stack After Interrupt\cite[p.~252]{AMD64Vol2}}
\label{fig:amd64-long-mode-interrupt-stac}
\end{figure}

For a full context-switch, the other registers that are part of the context need to be handled by the \gls{os}'s interrupt function.

\chapter{Weaknesses That Affect Memory-Safety}
\label{context::weaknesses-mem-safety}
The previous \cref{context::introduction,context::os-dev-concepts} describe the concepts of memory management on \gls{amd64} and how mistakes might come into existence. 
This chapter describes the related software weaknesses which are too commonly found.
The underlying weakness classes are explained alongside real-world and exemplary manifestations in \gls{C}.
The latter are ported and compared to functionally equivalent versions written with \gls{Rust} in \cref{rnd::weakness-mitig-prev::porting-c-vulns}, 

%\section{\glsentrylong{CWE}}
%\label{context::weaknesses-mem-safety::cwe}
Ongoing effort of collecting, analyzing and classifying vulnerabilities and their underlying weaknesses has been expended by the \textit{The MITRE Corporation} in form of the \gls{CWE}.
It has grown to a large relational database of typed weaknesses.
The following information is provided for enumerations of the type weakness class:
\begin{markdown}
1. Description
1. Applicable Platforms
1. Common Consequences
1. Likelihood of Exploit
1. Demonstrative Examples
1. Potential Mitigations
1. Relationships
\end{markdown}

The relevant weaknesses for this study are children of the umbrella weakness \citetitle{MITRE-CWE-633}.
Their information about demonstrative examples and potential mitigations are relevant for this study.
This and the following \cref{context::weakness-mitigation} present this information for \gls{CWE-633} and selected children.

% TODO test the autocite command with footnotes
\section{\glsentrylong{CWE-119}}
\label{context::weaknesses-mem-safety::cwe::119}
One of its children weaknesses, \gls{CWE-119}, is particularly interesting.
Manifestations of this weakness are a direct violation of the memory-safety defined in \cref{context::introduction::memory-safety::def} must have occurred, which "can cause read or write operations to be performed on memory locations that may be associated with other variables, data structures, or internal program data.
As a result, an attacker may be able to execute arbitrary code, alter the intended control flow, read sensitive information, or cause the system to crash"\cite{MITRE-CWE-119}.
This can happen on certain languages, which "allow direct addressing of memory locations and do not automatically ensure that these locations are valid for the memory buffer that is being referenced.
\gls{C}, \gls{C++}, \gls{asm} and languages without memory management support"\autocite{MITRE-CWE-119}.
This formulation of languages prone to this weakness is incorrect, as it doesn't conform with the earlier statement of languages that "allow direct addressing of memory locations".
Direct memory addressing support doesn't imply a lack of memory management support.

There are languages that provide memory management support and still allow direct memory addressing, which is interesting for \gls{os} development.
\gls{Rust} is one of these languages, although it requires the developer to explicitly acknowledge all direct memory access operations with the \textit{unsafe} keyword.
More information on \gls{Rust} follows in \cref{rnd::rust}.

\section{Statistics}
\label{context::weaknesses-mem-safety::cwe::statistics}
One of the main reasons for me to work on this topic is the increasing number of vulnerabilities based on memory-safety issues.

This section is intended to express the weakness's severity in real-world software based on available statistics.
The only data available are based on publicly available sources, thus the completeness of it is questionable, because many organizations might choose to not disclose their vulnerabilities, either to protect their reputation or for security reasons as explained in \cref{context::introduction::memory-safety-violation-in-sw}.
The data and visualizations are supplied by the \gls{NVD}, which collects the data based on the umbrella weakness \gls{CWE-635} that was specifically created for the \gls{NVD}.
The numbers of these selected weaknesses are detailed in the following figures, the rest is grouped as \textit{other}.

\Cref{fig:vulnerability-ratio-history,fig:vulnerability-counts-history} display a decade of data on vulnerabilities grouped by their \gls{CWE} category.
The category called \textit{buffer\footnote{A bounded chunk of memory used by programs to store and exchange data} errors} \gls{CWE-119}. 
In \cref{fig:vulnerability-ratio-history} it has the color light blue, 2nd from the bottom in the legend, and in \cref{fig:vulnerability-counts-history} it has the color blue, 2nd from the top in the legend.

\begin{figure}
\centering
\includegraphics[width=\textwidth]{gfx/Relative-Vulnerability-Type-Totals-By-Year}
\caption{Vulnerability Relative Counts History\cite{NVD}}
\label{fig:vulnerability-ratio-history}
\includegraphics[width=\textwidth]{gfx/Vulnerability-Type-Change-by-Year}
\caption{Vulnerability Absolute Counts History\cite{NVD}}
\label{fig:vulnerability-counts-history}
\end{figure}

\begin{table}
    \centering
    \begin{spreadtab}{{tabular}{ c | c | c }}
        @ Year & @ \% & @ count \\
        \hline
        @ 2007 & 6.75  & 490 \\
        @ 2008 & 10.01 & 550 \\
        @ 2009 & 9.84  & 530 \\
        @ 2010 & 11.58 & 530 \\
        @ 2011 & 15.95 & 600 \\
        @ 2012 & 13.67 & 650 \\
        @ 2013 & 14.63 & 670 \\
        @ 2014 & 9.69  & 800 \\
        @ 2015 & 15.18 & 1050 \\
        @ 2016 & 18.46 & 1150 \\
        @ 2017 & 16.34 & @ - \\
        \hline
        @ Average & :={round(sum([0,-11]:[0,-1])/11, 2)} & @- \\
    \end{spreadtab}
    \caption{Vulnerability \textit{"buffer error"} Counts History}
    \label{tab:vulnerability-buffer-error-by-history}
\end{table}

In \cref{tab:vulnerability-buffer-error-by-history}, the column \textit{relative count} represents \cref{fig:vulnerability-ratio-history}, and the column \textit{absolute count} represents \cref{fig:vulnerability-counts-history}.
With 16.34 percent of all vulnerabilities known by 2016, and an average of 12.92 percent over ten years, \gls{CWE-119} makes up a significant part of real-world weaknesses.

\subsection{Vulnerable APIs in Linux and C/C++}
\label{context::weaknesses-mem-safety::vuln-apis-linux-c}
\glspl{api} are a ubiquitous for programmers to access all kinds of functionality, serving as interfaces to network services, providing existing algorithms in form of libraries and frameworks, or interfacing with the local \gls{os}.
It is inherently dangerous to expose any sort of functionality through an \gls{api}, as it might contain bugs that will be spread widely with rising popularity.
Every \gls{os} needs to provide an \gls{api} for it's core functionality to be useful and extendable.

A very popular and widely supported \gls{os} is \gls{LX}.
The system libraries and the kernel are written in \gls{C}, the latter containing some hardware specific \gls{asm} code.
\gls{LX} is very popular for embedded systems, network servers and large-scale computers. % TODO: reference
Through \gls{android}, \gls{LX} has been distributed to a huge amount of mobile devices within the last decade. % TODO: reference
The list of vulnerabilities that are found in \gls{LX} device drivers which were written by \gls{android} device vendors is very concerning.
Even though Device drivers are not necessarily complex per-se, as they essentially just copy data to and from the hardware they target.
The difficulty is to perform these transfers only under safe circumstances, which are not always straight forward to identify or simply forgotten.

\gls{LX} has a huge ecosystem with existing libraries for any imaginable use-case from cryptography to artificial intelligence to give random examples.
It is necessary to investigate some of the weakness manifestations in detail in order to analyze if these might be prevented by using \gls{Rust}.

\section{Manifestations}
\label{context::weaknesses-mem-safety::manifestations}
This section contains real-world manifestations and \textit{re}constructed experiments of memory-safety related weaknesses.
It requires common understanding of the \gls{C} language and knowledge from the previous chapters.

\subsection{Official CWE Examples}
\label{context::weaknesses-mem-safety::manifestations::cwe-ex}
The following examples are officially listed under various children of \citetitle{MITRE-CWE-633}\cite{MITRE-CWE-633}.
The code and descriptions are copied literally and are used as references throughout the document.
The numbering in the example names aren't contiguous because only a subset was selected, and the naming is supposed to match the \gls{CWE} website.
References to the respective \gls{Rust} version are generally found in \cpnameref{rnd::weakness-mitig-prev::porting-c-vulns}, and referenced under each specific example for easier navigation.

\subsubsection{\glsentrylong{CWE-119}}

\paragraph{Example 1}
The following code asks the user to enter their last name and then attempts to store the value entered in the last\_name array.

\begin{listing}[htb]
    \begin{minted}[autogobble,linenos,breaklines=true]{c}
    ...
   char last_name[20];
   printf ("Enter your last name: ");
   scanf ("%s", last_name);
    ...
    \end{minted}
    \caption{CWE-120: Example 1 Bad Code C}
    \label{code::CWE-120::Example1BadCCode}
\end{listing}
\FloatBarrier

The problem with the code above is that it does not restrict or limit the size of the name entered by the user.If the user enters "Very\_very\_long\_last\_name" which is 24 characters long, then a buffer overflow will occur since the array can only hold 20 characters total. 

% \paragraph{Example 1}
% This example takes an IP address from a user, verifies that it is well formed and then looks up the hostname and copies it into a buffer.
% 
% \begin{listing}[htb]
%     \begin{minted}[autogobble,linenos,breaklines=true]{c}
%     void host_lookup(char *user_supplied_addr){
%         struct hostent *hp;
%         in_addr_t *addr;
%         char hostname[64];
%         in_addr_t inet_addr(const char *cp);
% 
%         /*routine that ensures user_supplied_addr is in the right format for conversion */
%         validate_addr_form(user_supplied_addr);
%         addr = inet_addr(user_supplied_addr);
%         hp = gethostbyaddr( addr, sizeof(struct in_addr), AF_INET);
%         strcpy(hostname, hp->h_name);
%     }
%     \end{minted}
%     \caption{CWE-119: Example 1 Bad Code C}
%     \label{code::CWE-119::Example1BadCCode}
% \end{listing}
% 
% This function allocates a buffer of 64 bytes to store the hostname, however there is no guarantee that the hostname will not be larger than 64 bytes. If an attacker specifies an address which resolves to a very large hostname, then we may overwrite sensitive data or even relinquish control flow to the attacker.
% 
% Note that this example also contains an unchecked return value (CWE-252) that can lead to a NULL pointer dereference (CWE-476). 

\paragraph{Example 2}
This example applies an encoding procedure to an input string and stores it into a buffer.

\begin{listing}[htb]
    \begin{minted}[autogobble,linenos,breaklines=true]{c}
    char * copy_input(char *user_supplied_string){
        int i, dst_index;
        char *dst_buf = (char*)malloc(4*sizeof(char) * MAX_SIZE);
        if ( MAX_SIZE <= strlen(user_supplied_string) ){
            die("user string too long, die evil hacker!");
        }
        dst_index = 0;
        for ( i = 0; i < strlen(user_supplied_string); i++ ){
            if( '&' == user_supplied_string[i] ){
            dst_buf[dst_index++] = '&';
            dst_buf[dst_index++] = 'a';
            dst_buf[dst_index++] = 'm';
            dst_buf[dst_index++] = 'p';
            dst_buf[dst_index++] = ';';
            }
            else if ('<' == user_supplied_string[i] ){
            /* encode to &lt; */
            } else dst_buf[dst_index++] = user_supplied_string[i];
        }
        return dst_buf;
    }
    \end{minted}
    \caption{CWE-119: Example 2 Bad Code C}
    \label{code::CWE-119::Example2BadCodeC}
\end{listing}
\FloatBarrier
The programmer attempts to encode the ampersand character in the user-controlled string, however the length of the string is validated before the encoding procedure is applied. Furthermore, the programmer assumes encoding expansion will only expand a given character by a factor of 4, while the encoding of the ampersand expands by 5. As a result, when the encoding procedure expands the string it is possible to overflow the destination buffer if the attacker provides a string of many ampersands.

\paragraph{Example 3}
The following example asks a user for an offset into an array to select an item.

\begin{listing}[htb]
    \begin{minted}[autogobble,linenos,breaklines=true]{c}
    int main (int argc, char **argv) {
        char *items[] = {"boat", "car", "truck", "train"};
        int index = GetUntrustedOffset();
        printf("You selected %s\n", items[index-1]);
    }
    \end{minted}
    \caption{CWE-119: Example 3 Bad Code C}
    \label{code::CWE-119::Example3BadCodeC}
\end{listing}
\FloatBarrier
 The programmer allows the user to specify which element in the list to select, however an attacker can provide an out-of-bounds offset, resulting in a buffer over-read (CWE-126). 

\paragraph{Example 4}
In the following code, the method retrieves a value from an array at a specific array index location that is given as an input parameter to the method
\begin{listing}[htb]
    \begin{minted}[autogobble,linenos,breaklines=true]{c}
    int getValueFromArray(int *array, int len, int index) {

        int value;

        // check that the array index is less than the maximum
        // length of the array
        if (index < len) {

            // get the value at the specified index of the array
            value = array[index];
        }
        // if array index is invalid then output error message
        // and return value indicating error
        else {
            printf("Value is: %d\n", array[index]);
            value = -1;
        }

        return value;
    }
    \end{minted}
    \caption{CWE-119: Example 4 Bad Code C}
    \label{code::CWE-119::Example4BadCodeC}
\end{listing}
\FloatBarrier

However, this method only verifies that the given array index is less than the maximum length of the array but does not check for the minimum value (CWE-839). This will allow a negative value to be accepted as the input array index, which will result in a out of bounds read (CWE-125) and may allow access to sensitive memory. The input array index should be checked to verify that is within the maximum and minimum range required for the array (CWE-129). In this example the if statement should be modified to include a minimum range check, as shown below. 

\begin{listing}[htb]
    \begin{minted}[autogobble,linenos,breaklines=true]{c}
    ...

    // check that the array index is within the correct
    // range of values for the array
    if (index >= 0 && index < len) {

    ...
    \end{minted}
    \caption{CWE-119: Example 4 Good Code C}
    \label{code::CWE-119::Example4GoodCodeC}
\end{listing}
\FloatBarrier


\subsubsection{\glsentrylong{CWE-122}}

\paragraph{Example 1}

While buffer overflow examples can be rather complex, it is possible to have very simple, yet still exploitable, heap-based buffer overflows:
\begin{listing}[htb]
    \begin{minted}[autogobble,linenos,breaklines=true]{c}
    #define BUFSIZE 256
    int main(int argc, char **argv) {
        char *buf;
        buf = (char *)malloc(sizeof(char)*BUFSIZE);
        strcpy(buf, argv[1]);
    }
    \end{minted}
    \caption{CWE-120: Example 1 Bad Code C}
    \label{code::CWE-122::Example1BadodeC}
\end{listing}
\FloatBarrier
The buffer is allocated heap memory with a fixed size, but there is no guarantee the string in argv[1] will not exceed this size and cause an overflow. 

\subsubsection{\glsentrylong{CWE-134}}

\paragraph{Example 1}
The following program prints a string provided as an argument.
\begin{listing}[htb]
    \begin{minted}[autogobble,linenos,breaklines=true]{c}
    #include <stdio.h>

    void printWrapper(char *string) {
        printf(string);
    }

    int main(int argc, char **argv) {
        char buf[5012];
        memcpy(buf, argv[1], 5012);
        printWrapper(argv[1]);
        return (0);
    }
    \end{minted}
    \caption{CWE-134: Example 1 Bad Code C}
    \label{code::CWE-134::Example1BadodeC}
\end{listing}
\FloatBarrier
The example is exploitable, because of the call to printf() in the printWrapper() function. Note: The stack buffer was added to make exploitation more simple.

\subsection{Heartbleed}
% TODO: paper about hearbleed with Rust

\subsection{BlueBorne on Linux}
Implementation of complex protocols are extremely dangerous in C as there is no notion of iterating or collections. 
\gls{LX}, which is written in C, suffers extensively from buffer related errors and is likely to do so in the foreseeable future.
Many hardware drivers are implementers of such protocols by exchanging data with the hardware and verifying the content.
Often protocols aren't ideal either which makes their implementation even harder.

A very recent and high impact vulnerability group is code named BlueBorne\cite{Seri2017}.
It has multiple manifestations on various \gls{os}, including Android and \gls{LX}, e.g. CVE-2017-1000251\cite[p.12]{Seri2017} which is explained here.

The vulnerable code is quite long and staged over three functions, though the following should be enough to indicate the issue:

\begin{minted}[breaklines]{c}
...
    char buf [64];
    len = l2cap_parse_conf_rsp (chan, rsp -> data , len, buf , & result);
...
\end{minted}
\code{buf}, which is subject to overflow in this vulnerability, is a stack array with a fixed size, and its pointer is passed into a function.
Inside the function is a loop which writes \code{len} messages from \code{rsp->data} to \code{buf}, while the last two are both of the "type" \code{void *}.
The length to \code{buf} is not known in the function, thus there is no possibility of intentionally respecting its boundaries.

\citeauthor{Seri2017} has additional criticism about \gls{LX}'s choice of having this code in the kernel:\footnote{italicized text was added by me}
"L2CAP\textit{, which can be seen as Bluetooth’s equivalent of TCP,} is included as part of the core Linux kernel code.
This is a rather dangerous choice.
Combining a fully exposed communication protocol, arcane features like EFS and a kernel space implementation is a recipe for trouble.
This vulnerability is a classic stack overflow occurring in the context of a kernel thread.
This provides an attacker with a full and reliable kernel-level exploit for any Bluetooth enabled device running Linux, requiring no additional steps.
Moreover, each compromised host can be used to launch secondary attacks, making this vulnerability wormable."

\subsection{The Stack Clash}
\label{context::weaknesses-mem-safety::manifestations::stack-clash}
A recent high severity vulnerability named \textit{Stack Clash}\cite{TheStackClash}, is briefly described as \textit{"a vulnerability in the memory management of several operating systems. It affects Linux, OpenBSD, NetBSD, FreeBSD and Solaris, on i386 and amd64.  It can be exploited by attackers to corrupt memory and execute arbitrary code."}
The \gls{LX} specific vulnerability is listed as CVE-2017-1000364\footnote{\url{http://www.cvedetails.com/cve/CVE-2017-1000364/}}, where \textit{"an issue was discovered in the size of the stack guard page on Linux, specifically a 4k stack guard page is not sufficiently large and can be "jumped" over (the stack guard page is bypassed)"}.
The vulnerability is assigned to the \citetitle{MITRE-CWE-119}\autocite{MITRE-CWE-119} presented in \cref{context::weaknesses-mem-safety::cwe::119}.

\subsubsection{Affecting Multiple OSs}
The vulnerability is extremely interesting for several reasons.
The issue has been recognized in 2005 and partially fixed in \gls{LX} in 2010 by introducing a guard page, but it wasn't considered to be a high risk.
Several years later, It has been found to affect the memory management of several \glspl{os} which don't share the same code base.

The two affected architectures are closely related and share the same memory-paging concepts, most of which are explained in \cref{context::os-dev-concepts::hw-supported-mm::multilevel-paging-concept,context::os-dev-concepts::hw-supported-mm::multilevel-paging-amd64,context::os-dev-concepts::hw-supported-mm::page-fault}.
This simply answers the question how this vulnerability can be present on in multiple \glspl{os}; they all implemented the same concept when this vulnerability was not popular enough.

\subsubsection{Unguarded Stack Growth}
The guard page concept has been introduced to prevent the stack from growing further than it should.\footnote{Related Linux CVE-2010-2240}

If the stack grows contiguously and thus accesses \glspl{vaddr} that are not mapped to a page, the \gls{os}'s page-fault handler has the chance to inspect the situation.
The \gls{os} simply allocates a new page for the unmapped \gls{vaddr} and allows the process to grow its stack.
When the \gls{vaddr} accesses the defined guard page, the \gls{os} denies the operation and the process will be notified by a segmentation fault.
This works as long one of these conditions is true: 
\begin{itemize}
    \item The guard page spans a \gls{vaddr} range that is larger than the largest stack increment  
    \item The area behind the stack page is unmapped and will also a page-fault
\end{itemize}
As the existence of vulnerability proves, these two conditions aren't always met.

\subsubsection{The Stack and Heap Overlap}
The reason why the existing guard page didn't work as expected are cases were the stack can indeed increment by a large enough amount to leap over the guard page.
And somewhere beyond the guard page is the heap, and the stack can happen to grow large enough to reach it.
This effectively overlaps the stack and the heap and makes it possible to use access one through variables of the other variables.

The best case scenario is mere memory corruption and the crash of the application.
In the worst case there's possible execution of arbitrary code, which can be by used by an attacker to gain control of the application and possibly the whole system.

\subsubsection{Proposed Solutions}
\label{context::weaknesses-mem-safety::manifestations::stack-clash::proposals}

The researchers at \textit{qualys} suggest two solutions\cite[III]{TheStackClash}.

The first proposed solution is to increase the guard page to 1MB or larger within the \gls{os}.
This doesn't guarantee absolute protection as it still be possible to have a large enough stack growth that surpasses the 1MB guard.

The second solution doesn't involve the \gls{os} but is about the userspace programs.
The suggestion is to compile all \glspl{app} on the system with the \gls{GCC}\cite{GCC540} with the \code{-fstack-check} option.
This allegedly "prevents the stack-pointer from moving into another memory region without accessing the stack guard-page (it writes one word to every 4KB page allocated on the stack)."
Besides the mention of this option, there is no explanation of what the exact technical outcome is.
This is further investigated in \cref{rnd::weakness-mitig-prev::stack-protection}, as stack protection is also something procedures within the \gls{os} could make use of.

\chapter{Weakness Mitigation And Prevention Strategies}
\label{context::weakness-mitigation}
This chapter explains what can be done to mitigate and prevent software weaknesses, focusing on actions that can be taken by the developer.
The first step is to leverage the \gls{CWE}'s database with its suggestions.

\section{CWE-119 Mitigation Suggestions}
\label{context::weakness-mitigation::cwe-119-suggestions}
The \gls{CWE-119} lists mitigation attempts for these software life cycle phases: Requirements, Architecture and Design, Implementation, and Operation.

\paragraph{Requirements Phase - Choose Language That Avoids Weaknesses}
For the requirements phase, \gls{CWE-119} suggests to "use a language that does not allow this weakness to occur or provides constructs that make this weakness easier to avoid".
Such languages "perform their own memory management are not subject to buffer overflows".

This is followed by various other suggestions, of which the most relevant ones have been extracted into the following list.
This list serves as additional test criteria against \gls{Rust}, evaluated in \cref{rnd}.

\paragraph{Architecture and Design - Use Libraries or Frameworks}
Use a vetted library or framework that does not allow this weakness to occur or provides constructs that make this weakness easier to avoid.

These libraries provide safer versions of overflow-prone string-handling functions: Safe C String Library (SafeStr) 

\paragraph{Build and Compilation - Hardened Compilation}
Run or compile the software using features or extensions that automatically provide a protection mechanism that mitigates or eliminates buffer overflows.

Examples: FORTIFY\_SOURCE GCC flag

\paragraph{Implementation}
This refers to the phase during which the programmer writes code.
        
\subparagraph{Careful Buffer Handling}
\begin{itemize}
\item Double check that your buffer is as large as you specify.
\item When using functions that accept a number of bytes to copy, such as strncpy(), be aware that if the destination buffer size is equal to the source buffer size, it may not NULL-terminate the string.
\item Check buffer boundaries if accessing the buffer in a loop and make sure you are not in danger of writing past the allocated space.
\item If necessary, truncate all input strings to a reasonable length before passing them to the copy and concatenation functions.
\end{itemize}

\subparagraph{Use Functions That Support Bounds}
\begin{itemize}
    \item Replace unbounded copy functions with analogous functions that support length arguments, such as strcpy with strncpy.
    \item Create these if they are not available.
\end{itemize}

\paragraph{Operation - Address Randomization}
Run or compile the software using features or extensions that randomly arrange the positions of a program's executable and libraries in memory. Because this makes the addresses unpredictable, it can prevent an attacker from reliably jumping to exploitable code.

Examples: Address Space Layout Randomization (ASLR),  Position-Independent Executables (PIE)

\section{Choice of Language}
All of the suggestions provided by the \gls{CWE} are in one way or another related the to \gls{proglang}.
Depending on the stage of the software development cycle, the suggestion is to either chose a stronger -- in the sense of less prone to weaknesses -- or avoid dangerous elements of a weaker \gls{proglang}.
Combining this with the knowledge from previous chapters, it can be concluded that the choice of \gls{proglang} is significant for memory-safety in \gls{os} as well as \glspl{app}.
This study does not have to workaround the weaknesses of any languages but is on to evaluating a potentially stronger. 
Thus, the above information is another motivator for this study's topic.

The next \cnameref{context::os-dev-lang-choice} further drives this evaluation.

\chapter{OS Development: Choice of Programming Language}
\label{context::os-dev-lang-choice}
There are dozens of \glspl{proglang} to write \glspl{app}, but only a few are viable for writing \glspl{os}.

\section{Abstraction Trade Offs: Safety vs. Functionality}
\label{context::os-dev-lang-choice::abstraction-safety-functionality}
In computer systems, safety and functionality are counter-proportional towards each other, because with increased functionality also grows complexity, and error cases become more difficult to find.
Applying this analogy to \glspl{proglang} might be misleading at first.
It might seem that the more abstraction is provided by a language, the higher the available functionality is.
In fact, the opposite is the case.

Abstraction can be used to impose limits on what the programmer can instruct the system to do, which can reduce functionality and increase safety.
By defining an abstraction layer in form of a \gls{proglang}, the language defines which of the underlying functionality will be exposed through it.
The \gls{proglang} can be designed with obligated rules, that make the written source code easier to analyze in an automated fashion before it gets compiled to the underlying representation.

\section{Requirements}
\label{context::os-dev-lang-choice::requirements}
Criteria for the choice of \gls{proglang} are much different for an \gls{os} than for other types of \glspl{app}.

\subsection{Technical}
\begin{itemize}
    \item{Compiler that generates machine code}
    \item{Inline \gls{cpu} instruction calls}
    \item{Direct memory addressing}
    \item{No requirement on an internal runtime or virtual machine}
    \item{No dependency on platform functionality}
\end{itemize}

The above features are technical requirements to be able to produce code for the bare machine.

\subsection{Safety-Critical}
In addition, this study defines more theoretical requirements that help prevent memory-safety critical mistakes.

\begin{itemize}
    \item{Static analysis as part of the language design}
    \item{Memory-safety as a 1st-class feature}
    \item{Object oriented or similar type system that allows modeling of complex relations between types}
\end{itemize}
These features often come at the cost of a more complex language design, which results in a steep learning curve for beginners and might not seem worth it.

As explained in \cnameref{context::introduction::memory-safety::time-aspect}, mistakes need to be identified as soon as possible, and static analysis is the technical method for this.
The static analyzer can be designed to target different aspects and the language can be designed to offer these kind of aspects to the static analyzer.
For this study the primary aspect is memory-safety, which is assisted by the type system.
For a simple example, a static analyzer should detect casts between incompatible pointer types, as it will eventually lead to undesired effects.

\section{Why not \glsentrytext{C}}
\gls{C} fulfills all these requirements, and since it's development it has replaced Assembly as the de-facto standard \gls{proglang} for writing \glspl{os}.
As discussed in \cref{context::weaknesses-mem-safety} it has dangerous weaknesses.
With the growing number of vulnerabilities, various solutions have been proposed to increase the safety of C, either with static code analysis or via \gls{compiler}-generated code that imposes runtime checks.

Checks that are performed at runtime introduce a high degree of overhead, which makes it a nonviable option in the domain of \gls{os} development, where many code paths must be very fast to ensure the operation of high speed I/O devices\cite[p.~1]{Balasubramanian2017} or tasks with \gls{realtime} requirements.
This has been forcing many developers to prioritize performance over safety, and others to look for alternatives.

C allows direct access to memory via pointers and arithmetic thereof.
Static source code analysis is difficult on \gls{C}, as it that has not been designed for this purpose and has too many ambiguities.\cite{Kowshik2002}
C has no notion of objects beyond structs, which makes more complex memory structures, e.g. double-ended-queues trees, a hurdle to write and maintain safely.
As recorded by the \gls{CWE} and shown in \cref{context::weaknesses-mem-safety::manifestations::cwe-ex}, even access to simple arrays is often mishandled.
These criteria rule out \gls{C} for the purpose of this study.

There have been attempts to define subsets of the \gls{C} language that can be safety checked, e.g. Cyclone\cite{Jim2002}, Control-C\cite{Kowshik2002}, but none has managed to take C's position.
\paragraph{Control-C} 
Control-C is created on top of \gls{llvm} as a statically provable memory-safe strict subset of C, purposed for real-time control systems.\cite{Kowshik2002}.
Hardware requirements for handling safe runtime errors, which are known as \gls{cpu} exceptions on \gls{amd64}.
In Control-C they could be caused by stack and heap overflow, stack overflow (e.g., due to infinite recursion) and heap overflow due to dynamic allocation.
The latter sounds similar to the concept of guard pages and page-faults described in \cref{context::weaknesses-mem-safety::manifestations::stack-clash,rnd::weakness-mitig-prev::stack-protection::stack-clash}.
Control-C and Rust have certain similarities, e.g. both and aim to guarantee memory-safety, use static type checks and make use of LLVM as the backend.
Control-C uses affine transformation rules to statically guarantee bounds on array operations, which Rust does not do.
It can only be speculated why Control-C hasn't gained popularity since its invention.
It may be possible that it was mislabeled as a real-time only system language and was neglected in more generic language research.

From today's perspective it appears to be clearly inferior to younger \gls{os} language candidates like Rust, e.g. as it is a subset of C and thus doesn't support object oriented paradigms.

\section{LLVM: C and Rust}
The \glsentrydesc{llvm} has become popular since it was introduced.
To demonstrate this, \cref{fig:llvm-research-papers-since-2002}\footnote{\url{TODO https://llvm.org/pubs/}} shows a statistic on how many research papers have been published yearly since 2002.

\begin{figure}[ht!]
    \centering
    \includegraphics[width=0.6\textwidth]{gfx/llvm-number-paper-pa.png}
    \caption{Research Papers About LLVM since 2002}
    \label{fig:llvm-research-papers-since-2002}
\end{figure}
\FloatBarrier

It has since become a viable backend for the \gls{clang} \gls{compiler} and is also used by \gls{Rust}.

The fronted \glspl{compiler} produce code for the \gls{llvm} Intermediate-Represantion, which is a hardware-independent.
\gls{llvm} can produce machine code for supported architectures, a list of which is found on the official website \footnote{\url{https://llvm.org/docs/CompilerWriterInfo.html\#hardware}}.
It contains all contemporary architectures, including \gls{amd64}.

\section{Why Investigate Rust}
Rust has gained popularity in among hobby \gls{os} developers.
Reasons for this vary from simply challenging the possibility due to being convinced about its claimed features to guarantee memory-safety.
Theese officially advertised features are\footnote{\url{https://www.rust-lang.org/}}
\begin{itemize}
    \item zero-cost abstractions
    \item move semantics
    \item guaranteed memory safety
    \item threads without data races
    \item trait-based generics
    \item pattern matching
    \item type inference
    \item minimal runtime
    \item efficient C bindings.
\end{itemize}

Research articles and existing projects that have appeared in recent months and years have contributed motivation and information to this study.
Some of them are listed in \cref{rnd::existing-os-dev-with-rust::papers} next to the existing projects.
Others are referenced throughout the various chapters of \cref{rnd}. 
They have all been motivators and plenty of justification for this study to pay closer attention to Rust.

\section{Next Steps}
At this point, the assumption is made that \gls{Rust} can increase memory-safety in the \gls{os} in comparison to using \gls{C}.
In the next \cnameref{rnd} it is studied which of the above features contribute to memory-safety in the \gls{os}, and the corresponding language items are identified.
