tjh.dev
Linux kernel mirror (for testing)
git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel
os
linux
1The PPC KVM paravirtual interface
2=================================
3
4The basic execution principle by which KVM on PowerPC works is to run all kernel
5space code in PR=1 which is user space. This way we trap all privileged
6instructions and can emulate them accordingly.
7
8Unfortunately that is also the downfall. There are quite some privileged
9instructions that needlessly return us to the hypervisor even though they
10could be handled differently.
11
12This is what the PPC PV interface helps with. It takes privileged instructions
13and transforms them into unprivileged ones with some help from the hypervisor.
14This cuts down virtualization costs by about 50% on some of my benchmarks.
15
16The code for that interface can be found in arch/powerpc/kernel/kvm*
17
18Querying for existence
19======================
20
21To find out if we're running on KVM or not, we leverage the device tree. When
22Linux is running on KVM, a node /hypervisor exists. That node contains a
23compatible property with the value "linux,kvm".
24
25Once you determined you're running under a PV capable KVM, you can now use
26hypercalls as described below.
27
28KVM hypercalls
29==============
30
31Inside the device tree's /hypervisor node there's a property called
32'hypercall-instructions'. This property contains at most 4 opcodes that make
33up the hypercall. To call a hypercall, just call these instructions.
34
35The parameters are as follows:
36
37 Register IN OUT
38
39 r0 - volatile
40 r3 1st parameter Return code
41 r4 2nd parameter 1st output value
42 r5 3rd parameter 2nd output value
43 r6 4th parameter 3rd output value
44 r7 5th parameter 4th output value
45 r8 6th parameter 5th output value
46 r9 7th parameter 6th output value
47 r10 8th parameter 7th output value
48 r11 hypercall number 8th output value
49 r12 - volatile
50
51Hypercall definitions are shared in generic code, so the same hypercall numbers
52apply for x86 and powerpc alike with the exception that each KVM hypercall
53also needs to be ORed with the KVM vendor code which is (42 << 16).
54
55Return codes can be as follows:
56
57 Code Meaning
58
59 0 Success
60 12 Hypercall not implemented
61 <0 Error
62
63The magic page
64==============
65
66To enable communication between the hypervisor and guest there is a new shared
67page that contains parts of supervisor visible register state. The guest can
68map this shared page using the KVM hypercall KVM_HC_PPC_MAP_MAGIC_PAGE.
69
70With this hypercall issued the guest always gets the magic page mapped at the
71desired location. The first parameter indicates the effective address when the
72MMU is enabled. The second parameter indicates the address in real mode, if
73applicable to the target. For now, we always map the page to -4096. This way we
74can access it using absolute load and store functions. The following
75instruction reads the first field of the magic page:
76
77 ld rX, -4096(0)
78
79The interface is designed to be extensible should there be need later to add
80additional registers to the magic page. If you add fields to the magic page,
81also define a new hypercall feature to indicate that the host can give you more
82registers. Only if the host supports the additional features, make use of them.
83
84The magic page has the following layout as described in
85arch/powerpc/include/asm/kvm_para.h:
86
87struct kvm_vcpu_arch_shared {
88 __u64 scratch1;
89 __u64 scratch2;
90 __u64 scratch3;
91 __u64 critical; /* Guest may not get interrupts if == r1 */
92 __u64 sprg0;
93 __u64 sprg1;
94 __u64 sprg2;
95 __u64 sprg3;
96 __u64 srr0;
97 __u64 srr1;
98 __u64 dar;
99 __u64 msr;
100 __u32 dsisr;
101 __u32 int_pending; /* Tells the guest if we have an interrupt */
102};
103
104Additions to the page must only occur at the end. Struct fields are always 32
105or 64 bit aligned, depending on them being 32 or 64 bit wide respectively.
106
107Magic page features
108===================
109
110When mapping the magic page using the KVM hypercall KVM_HC_PPC_MAP_MAGIC_PAGE,
111a second return value is passed to the guest. This second return value contains
112a bitmap of available features inside the magic page.
113
114The following enhancements to the magic page are currently available:
115
116 KVM_MAGIC_FEAT_SR Maps SR registers r/w in the magic page
117
118For enhanced features in the magic page, please check for the existence of the
119feature before using them!
120
121MSR bits
122========
123
124The MSR contains bits that require hypervisor intervention and bits that do
125not require direct hypervisor intervention because they only get interpreted
126when entering the guest or don't have any impact on the hypervisor's behavior.
127
128The following bits are safe to be set inside the guest:
129
130 MSR_EE
131 MSR_RI
132 MSR_CR
133 MSR_ME
134
135If any other bit changes in the MSR, please still use mtmsr(d).
136
137Patched instructions
138====================
139
140The "ld" and "std" instructions are transormed to "lwz" and "stw" instructions
141respectively on 32 bit systems with an added offset of 4 to accommodate for big
142endianness.
143
144The following is a list of mapping the Linux kernel performs when running as
145guest. Implementing any of those mappings is optional, as the instruction traps
146also act on the shared page. So calling privileged instructions still works as
147before.
148
149From To
150==== ==
151
152mfmsr rX ld rX, magic_page->msr
153mfsprg rX, 0 ld rX, magic_page->sprg0
154mfsprg rX, 1 ld rX, magic_page->sprg1
155mfsprg rX, 2 ld rX, magic_page->sprg2
156mfsprg rX, 3 ld rX, magic_page->sprg3
157mfsrr0 rX ld rX, magic_page->srr0
158mfsrr1 rX ld rX, magic_page->srr1
159mfdar rX ld rX, magic_page->dar
160mfdsisr rX lwz rX, magic_page->dsisr
161
162mtmsr rX std rX, magic_page->msr
163mtsprg 0, rX std rX, magic_page->sprg0
164mtsprg 1, rX std rX, magic_page->sprg1
165mtsprg 2, rX std rX, magic_page->sprg2
166mtsprg 3, rX std rX, magic_page->sprg3
167mtsrr0 rX std rX, magic_page->srr0
168mtsrr1 rX std rX, magic_page->srr1
169mtdar rX std rX, magic_page->dar
170mtdsisr rX stw rX, magic_page->dsisr
171
172tlbsync nop
173
174mtmsrd rX, 0 b <special mtmsr section>
175mtmsr rX b <special mtmsr section>
176
177mtmsrd rX, 1 b <special mtmsrd section>
178
179[Book3S only]
180mtsrin rX, rY b <special mtsrin section>
181
182[BookE only]
183wrteei [0|1] b <special wrteei section>
184
185
186Some instructions require more logic to determine what's going on than a load
187or store instruction can deliver. To enable patching of those, we keep some
188RAM around where we can live translate instructions to. What happens is the
189following:
190
191 1) copy emulation code to memory
192 2) patch that code to fit the emulated instruction
193 3) patch that code to return to the original pc + 4
194 4) patch the original instruction to branch to the new code
195
196That way we can inject an arbitrary amount of code as replacement for a single
197instruction. This allows us to check for pending interrupts when setting EE=1
198for example.