KVM is one of several virtualization technologies available for Linux. KVM generally requires hardware support but can run an unmodified OS image but in exchange offers full system virtualization, good performance. Also since KVM is based on a standard Linux kernel the host system uses the standard I/O drivers available for the platform.
Trap and Emulate
KymaSys implemented MIPS support in KVM for MIPS32r2 processors using Trap and Emulate, which was merged in Linux 3.10. The guest runs entirely in user mode which means in confined to the lowest 2GB of the address space. As the result the address space for guest applications is shrunk so guest kernels now use a 1GB/1GB userspace/kernelspace split. Privileged instructions trap to the host kernel which emulates the instruction.
Kyma claims “We've limited the changes to the Linux guest to less than 10 lines of code and use advanced techniques like run-time binary translation to minimize the number of traps and greatly improve performance.” for their software-only implementation. If their technology holds up to its promise they should deliver performance below but fairly close to that of a hardware-based KVM implementation.
The Trap & Emulate version of KVM will expose a non-traditional set of segments to the guest:
|Name||Privilege Level||Attributes||Address Range||Size||Comments|
|KSeg3||Kernel||TLB||0xE0000000 .. 0xFFFFFFFF||512MB||Linux: vmalloc / modules|
|KSeg2||Supervisor||TLB||0xC0000000 .. 0xDFFFFFFF||512MB|
|KSeg1||Kernel||Unmapped, Uncached||0xA0000000 .. 0xBFFFFFFF||512MB||Linux: MMIO|
|KSeg0||Unmapped, Cached||0x80000000 .. 0x9FFFFFFF||512MB||Linux: Static code / data|
|USeg||Guest KSeg2||User|| Guest
|TLB||Guest TLB||0x60000000 .. 0x7FFFFFFF||2GB||512MB||Linux Guest: vmalloc / modules|
|Guest KSeg0||Guest Unmapped, Cached||0x40000000 .. 0x5FFFFFFF||512MB||Linux Guest: static code / data|
|Guest USeg|| Guest
|Guest TLB||0x00000000 .. 0x3FFFFFFF||1GB|
A Trap & Emulate guest kernel must be built with
CONFIG_KVM_GUEST=y, so that it is linked at
0x40000000 (Guest KSeg0) instead of the usual
VZ (Hardware Assisted Virtualization)
It modifies the existing exception handlers to handle guest mode, and is largely focussed on MIPS64 so far. David Daney has also ported the kvm tool to MIPS as the controlling program, so this implementation makes use of paravirtualization.
It creates a new exception vector (using the EBASE COP0 register) for use when running guests, and is largely focussed on MIPS32 so far.
Notable Features by Kernel Version
- Linux 3.10
- Initial Trap & Emulate support for MIPS32r2 processors.
- Linux 3.14
- Support TLBInv capable cores with EntryHi.EHInv.
- Linux 3.16
- Rewrite guest timer emulation with new user API, allowing precise control of guest timer.
- Support XBurst cores and <16K pages.
- Expose CP0_EPC, CP0_Count, CP0_Compare, CP0_UserLocal, CP0_HWREna registers to user API.
- Linux 4.1
- Expose FPU & MSA to guest.
- Expose CP0_PRid, FPU & MSA registers and writing of CP0_Config1, CP0_Config2, CP0_Config3, CP0_Config4, CP0_Config5 registers to user API.
- Handle host MSA Disabled & Trap exceptions.
- Linux 4.7
- MIPS groundwork for probing and accessing VZ hardware resources.
- Linux 4.8
- Support MIPS32r6 host & guest.
- Make KVM_GET_REG_LIST dynamic, and list FPU & MSA registers.
- Make Config3.ULRI writable by user API (to control exposure of CP0_UserLocal register).
- Emulate CP0_KScratch registers and expose to user.
- Move commpage away from 0 if possible to allow guest kernel NULL pointer dereferences to be handled by guest.
- Convert guest exception entry code to use UASM (dynamically generated).
- Add trace events: kvm_aux (FPU & MSA context management), kvm_asid_change, kvm_enter, kvm_reenter, kvm_hwr (guest register emulation).
- Pending for Linux 4.9
- Support MIPS64 hosts (with MIPS32 guest).
- Some support for Highmem host kernels.
This section details some future improvements that could be made (or have yet to be fully upstreamed) to KVM for MIPS:
- Memory management
- Convert GPA map from linear array to page table (in progress).
- Create Trap & Emulate GVA->HPA page table to avoid guest TLB lookups, and enable HTW to use it.
- Emulate guest FTLB to speed up guest TLB lookups (set associative) and allow much larger guest TLB.
- Create fast path TLB refill handler.
- VZ (Hardware Assisted Virtualization) support (in progress).
- EVA host support.
- Further Trap & Emulate optimisations.