summaryrefslogtreecommitdiffstats
path: root/arch/x86
Commit message (Collapse)AuthorAgeFilesLines
* KVM: SVM: Don't allow nested guest to VMMCALL into hostJoerg Roedel2010-07-051-2/+7
| | | | | | | | | | | | This patch disables the possibility for a l2-guest to do a VMMCALL directly into the host. This would happen if the l1-hypervisor doesn't intercept VMMCALL and the l2-guest executes this instruction. Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de> (cherry picked from commit 0d945bd9351199744c1e89d57a70615b6ee9f394)
* KVM: x86: Inject #GP with the right rip on efer writesRoedel, Joerg2010-07-051-19/+12
| | | | | | | | | | | | | | | This patch fixes a bug in the KVM efer-msr write path. If a guest writes to a reserved efer bit the set_efer function injects the #GP directly. The architecture dependent wrmsr function does not see this, assumes success and advances the rip. This results in a #GP in the guest with the wrong rip. This patch fixes this by reporting efer write errors back to the architectural wrmsr function. Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de> (cherry picked from commit b69e8caef5b190af48c525f6d715e7b7728a77f6)
* KVM: x86: Add missing locking to arch specific vcpu ioctlsAvi Kivity2010-07-051-0/+6
| | | | | | Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de> (cherry picked from commit 8fbf065d625617bbbf6b72d5f78f84ad13c8b547)
* KVM: Fix wallclock version writing raceAvi Kivity2010-07-051-2/+10
| | | | | | | | | | | Wallclock writing uses an unprotected global variable to hold the version; this can cause one guest to interfere with another if both write their wallclock at the same time. Acked-by: Glauber Costa <glommer@redhat.com> Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de> (cherry picked from commit 9ed3c444ab8987c7b219173a2f7807e3f71e234e)
* KVM: MMU: Don't read pdptrs with mmu spinlock held in mmu_alloc_rootsAvi Kivity2010-07-051-0/+9
| | | | | | | | | | | | On svm, kvm_read_pdptr() may require reading guest memory, which can sleep. Push the spinlock into mmu_alloc_roots(), and only take it after we've read the pdptr. Tested-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de> (cherry picked from commit 8facbbff071ff2b19268d3732e31badc60471e21)
* KVM: VMX: enable VMXON check with SMX enabled (Intel TXT)Shane Wang2010-07-053-13/+25
| | | | | | | | | | | | | | | | Per document, for feature control MSR: Bit 1 enables VMXON in SMX operation. If the bit is clear, execution of VMXON in SMX operation causes a general-protection exception. Bit 2 enables VMXON outside SMX operation. If the bit is clear, execution of VMXON outside SMX operation causes a general-protection exception. This patch is to enable this kind of check with SMX for VMXON in KVM. Signed-off-by: Shane Wang <shane.wang@intel.com> Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de> (cherry picked from commit cafd66595d92591e4bd25c3904e004fc6f897e2d)
* KVM: MMU: Segregate shadow pages with different cr0.wpAvi Kivity2010-07-052-1/+3
| | | | | | | | | | | | | | When cr0.wp=0, we may shadow a gpte having u/s=1 and r/w=0 with an spte having u/s=0 and r/w=1. This allows excessive access if the guest sets cr0.wp=1 and accesses through this spte. Fix by making cr0.wp part of the base role; we'll have different sptes for the two cases and the problem disappears. Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de> (cherry picked from commit 3dbe141595faa48a067add3e47bba3205b79d33c)
* KVM: x86: Check LMA bit before set_eferSheng Yang2010-07-051-2/+2
| | | | | | | | | | kvm_x86_ops->set_efer() would execute vcpu->arch.efer = efer, so the checking of LMA bit didn't work. Signed-off-by: Sheng Yang <sheng@linux.intel.com> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de> (cherry picked from commit a3d204e28579427609c3d15d2310127ebaa47d94)
* KVM: Don't allow lmsw to clear cr0.peAvi Kivity2010-07-051-1/+1
| | | | | | | | | | | | The current lmsw implementation allows the guest to clear cr0.pe, contrary to the manual, which breaks EMM386.EXE. Fix by ORing the old cr0.pe with lmsw's operand. Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de> (cherry picked from commit f78e917688edbf1f14c318d2e50dc8e7dad20445)
* x86, paravirt: Add a global synchronization point for pvclockGlauber Costa2010-07-051-0/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In recent stress tests, it was found that pvclock-based systems could seriously warp in smp systems. Using ingo's time-warp-test.c, I could trigger a scenario as bad as 1.5mi warps a minute in some systems. (to be fair, it wasn't that bad in most of them). Investigating further, I found out that such warps were caused by the very offset-based calculation pvclock is based on. This happens even on some machines that report constant_tsc in its tsc flags, specially on multi-socket ones. Two reads of the same kernel timestamp at approx the same time, will likely have tsc timestamped in different occasions too. This means the delta we calculate is unpredictable at best, and can probably be smaller in a cpu that is legitimately reading clock in a forward ocasion. Some adjustments on the host could make this window less likely to happen, but still, it pretty much poses as an intrinsic problem of the mechanism. A while ago, I though about using a shared variable anyway, to hold clock last state, but gave up due to the high contention locking was likely to introduce, possibly rendering the thing useless on big machines. I argue, however, that locking is not necessary. We do a read-and-return sequence in pvclock, and between read and return, the global value can have changed. However, it can only have changed by means of an addition of a positive value. So if we detected that our clock timestamp is less than the current global, we know that we need to return a higher one, even though it is not exactly the one we compared to. OTOH, if we detect we're greater than the current time source, we atomically replace the value with our new readings. This do causes contention on big boxes (but big here means *BIG*), but it seems like a good trade off, since it provide us with a time source guaranteed to be stable wrt time warps. After this patch is applied, I don't see a single warp in time during 5 days of execution, in any of the machines I saw them before. Signed-off-by: Glauber Costa <glommer@redhat.com> Acked-by: Zachary Amsden <zamsden@redhat.com> CC: Jeremy Fitzhardinge <jeremy@goop.org> CC: Avi Kivity <avi@redhat.com> CC: Marcelo Tosatti <mtosatti@redhat.com> CC: Zachary Amsden <zamsden@redhat.com> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de> (cherry picked from commit 489fb490dbf8dab0249ad82b56688ae3842a79e8)
* KVM: SVM: Report emulated SVM features to userspaceJoerg Roedel2010-07-051-0/+10
| | | | | | | | | | | | | This patch implements the reporting of the emulated SVM features to userspace instead of the real hardware capabilities. Every real hardware capability needs emulation in nested svm so the old behavior was broken. Cc: stable@kernel.org Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de> (cherry picked from commit c2c63a493924e09a1984d1374a0e60dfd54fc0b0)
* KVM: x86: Add callback to let modules decide over some supported cpuid bitsJoerg Roedel2010-07-054-0/+15
| | | | | | | | | | | | | | This patch adds the get_supported_cpuid callback to kvm_x86_ops. It will be used in do_cpuid_ent to delegate the decission about some supported cpuid bits to the architecture modules. Cc: stable@kernel.org Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de> (cherry picked from commit d4330ef2fb2236a1e3a176f0f68360f4c0a8661b)
* KVM: SVM: Fix wrong interrupt injection in enable_irq_windowsJoerg Roedel2010-07-051-9/+8
| | | | | | | | | | | | | The nested_svm_intr() function does not execute the vmexit anymore. Therefore we may still be in the nested state after that function ran. This patch changes the nested_svm_intr() function to return wether the irq window could be enabled. Cc: stable@kernel.org Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de> (cherry picked from commit 8fe546547cf6857a9d984bfe2f2194910f3fc5d0)
* KVM: SVM: Don't sync nested cr8 to lapic and backJoerg Roedel2010-07-051-15/+31
| | | | | | | | | | | | | | | | | | This patch makes syncing of the guest tpr to the lapic conditional on !nested. Otherwise a nested guest using the TPR could freeze the guest. Another important change this patch introduces is that the cr8 intercept bits are no longer ORed at vmrun emulation if the guest sets VINTR_MASKING in its VMCB. The reason is that nested cr8 accesses need alway be handled by the nested hypervisor because they change the shadow version of the tpr. Cc: stable@kernel.org Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de> (cherry picked from commit 88ab24adc7142506c8583ac36a34fa388300b750)
* KVM: SVM: Fix nested msr intercept handlingJoerg Roedel2010-07-051-10/+3
| | | | | | | | | | | | The nested_svm_exit_handled_msr() function maps only one page of the guests msr permission bitmap. This patch changes the code to use kvm_read_guest to fix the bug. Cc: stable@kernel.org Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de> (cherry picked from commit 4c7da8cb43c09e71a405b5aeaa58a1dbac3c39e9)
* KVM: SVM: Sync all control registers on nested vmexitJoerg Roedel2010-07-051-0/+4
| | | | | | | | | | | | | | | Currently the vmexit emulation does not sync control registers were the access is typically intercepted by the nested hypervisor. But we can not count on that intercepts to sync these registers too and make the code architecturally more correct. Cc: stable@kernel.org Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de> (cherry picked from commit cdbbdc1210223879450555fee04c29ebf116576b)
* KVM: SVM: Fix schedule-while-atomic on nested exception handlingJoerg Roedel2010-07-051-4/+19
| | | | | | | | | | | | Move the actual vmexit routine out of code that runs with irqs and preemption disabled. Cc: stable@kernel.org Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de> (cherry picked from commit b8e88bc8ffba5fe53fb8d8a0a4be3bbcffeebe56)
* KVM: SVM: Don't use kmap_atomic in nested_svm_mapJoerg Roedel2010-07-051-23/+24
| | | | | | | | | | | | | | Use of kmap_atomic disables preemption but if we run in shadow-shadow mode the vmrun emulation executes kvm_set_cr3 which might sleep or fault. So use kmap instead for nested_svm_map. Cc: stable@kernel.org Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de> (cherry picked from commit 7597f129d8b6799da7a264e6d6f7401668d3a36d)
* perf_events: Fix resource leak in x86 __hw_perf_event_init()Stephane Eranian2010-07-051-1/+4
| | | | | | | | | | | | | | | | | | | | | commit 4b24a88b35e15e04bd8f2c5dda65b5dc8ebca05f upstream. If reserve_pmc_hardware() succeeds but reserve_ds_buffers() fails, then we need to release_pmc_hardware. It won't be done by the destroy() callback because we return before setting it in case of error. Signed-off-by: Stephane Eranian <eranian@google.com> Cc: peterz@infradead.org Cc: paulus@samba.org Cc: davem@davemloft.net Cc: fweisbec@gmail.com Cc: robert.richter@amd.com Cc: perfmon2-devel@lists.sf.net LKML-Reference: <4ba1568b.15185e0a.182a.7802@mx.google.com> Signed-off-by: Ingo Molnar <mingo@elte.hu> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* arch/x86/kernel: Add missing spin_unlockJulia Lawall2010-07-051-3/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | commit 84fe6c19e4a598e8071e3bd1b2c923454eae1268 upstream. Add a spin_unlock missing on the error path. The locks and unlocks are balanced in other functions, so it seems that the same should be the case here. The semantic match that finds this problem is as follows: (http://coccinelle.lip6.fr/) // <smpl> @@ expression E1; @@ * spin_lock(E1,...); <+... when != E1 if (...) { ... when != E1 * return ...; } ...+> * spin_unlock(E1,...); // </smpl> Signed-off-by: Julia Lawall <julia@diku.dk> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* xen: ensure timer tick is resumed even on CPU driving the resumeIan Campbell2010-07-051-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | commit cd52e17ea8278f8449b6174a8e5ed439a2e44ffb upstream. The core suspend/resume code is run from stop_machine on CPU0 but parts of the suspend/resume machinery (including xen_arch_resume) are run on whichever CPU happened to schedule the xenwatch kernel thread. As part of the non-core resume code xen_arch_resume is called in order to restart the timer tick on non-boot processors. The boot processor itself is taken care of by core timekeeping code. xen_arch_resume uses smp_call_function which does not call the given function on the current processor. This means that we can end up with one CPU not receiving timer ticks if the xenwatch thread happened to be scheduled on CPU > 0. Use on_each_cpu instead of smp_call_function to ensure the timer tick is resumed everywhere. Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Jeremy Fitzhardinge <jeremy@goop.org> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* x86, setup: Phoenix BIOS fixup is needed on Dell Inspiron Mini 1012Gabor Gombas2010-07-051-0/+11
| | | | | | | | | | | | | | | | | | commit 3d6e77a3ddb8e4156b89f4273ff8c7d37abaf781 upstream. The low-memory corruption checker triggers during suspend/resume, so we need to reserve the low 64k. Don't be fooled that the BIOS identifies itself as "Dell Inc.", it's still Phoenix BIOS. [ hpa: I think we blacklist almost every BIOS in existence. We should either change this to a whitelist or just make it unconditional. ] Signed-off-by: Gabor Gombas <gombasg@digikabel.hu> LKML-Reference: <201005241913.o4OJDIMM010877@imap1.linux-foundation.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: H. Peter Anvin <hpa@linux.intel.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* x86/amd-iommu: Fall back to GART if initialization failsJoerg Roedel2010-07-052-4/+9
| | | | | | | | | | | | | | commit d7f0776975334070a93370ae048fda0c31a91c38 upstream. This patch implements a fallback to the GART IOMMU if this is possible and the AMD IOMMU initialization failed. Otherwise the fallback would be nommu which is very problematic on machines with more than 4GB of memory or swiotlb which hurts io-performance. Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* x86/amd-iommu: Fix crash when request_mem_region failsJoerg Roedel2010-07-051-3/+8
| | | | | | | | | | | | | commit e82752d8b5a7e0a5e4d607fd8713549e2a4e2741 upstream. When request_mem_region fails the error path tries to disable the IOMMUs. This accesses the mmio-region which was not allocated leading to a kernel crash. This patch fixes the issue. Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* oprofile/x86: fix uninitialized counter usage during cpu hotplugRobert Richter2010-07-051-2/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | commit 2623a1d55a6260c855e1f6d1895900b50b40a896 upstream. This fixes a NULL pointer dereference that is triggered when taking a cpu offline after oprofile was initialized, e.g.: $ opcontrol --init $ opcontrol --start-daemon $ opcontrol --shutdown $ opcontrol --deinit $ echo 0 > /sys/devices/system/cpu/cpu1/online See the crash dump below. Though the counter has been disabled the cpu notifier is still active and trying to use already freed counter data. This fix is for linux-stable. To proper fix this, the hotplug code must be rewritten. Thus I will leave a WARN_ON_ONCE() message with this patch. BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffff8132ad57>] op_amd_stop+0x2d/0x8e PGD 0 Oops: 0000 [#1] SMP last sysfs file: /sys/devices/system/cpu/cpu1/online CPU 1 Modules linked in: Pid: 0, comm: swapper Not tainted 2.6.34-rc5-oprofile-x86_64-standard-00210-g8c00f06 #16 Anaheim/Anaheim RIP: 0010:[<ffffffff8132ad57>] [<ffffffff8132ad57>] op_amd_stop+0x2d/0x8e RSP: 0018:ffff880001843f28 EFLAGS: 00010006 RAX: 0000000000000000 RBX: 0000000000000000 RCX: dead000000200200 RDX: ffff880001843f68 RSI: dead000000100100 RDI: 0000000000000000 RBP: ffff880001843f48 R08: 0000000000000000 R09: ffff880001843f08 R10: ffffffff8102c9a5 R11: ffff88000184ea80 R12: 0000000000000000 R13: ffff88000184f6c0 R14: 0000000000000000 R15: 0000000000000000 FS: 00007fec6a92e6f0(0000) GS:ffff880001840000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 000000000163b000 CR4: 00000000000006e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process swapper (pid: 0, threadinfo ffff88042fcd8000, task ffff88042fcd51d0) Stack: ffff880001843f48 0000000000000001 ffff88042e9f7d38 ffff880001843f68 <0> ffff880001843f58 ffffffff8132a602 ffff880001843f98 ffffffff810521b3 <0> ffff880001843f68 ffff880001843f68 ffff880001843f88 ffff88042fcd9fd8 Call Trace: <IRQ> [<ffffffff8132a602>] nmi_cpu_stop+0x21/0x23 [<ffffffff810521b3>] generic_smp_call_function_single_interrupt+0xdf/0x11b [<ffffffff8101804f>] smp_call_function_single_interrupt+0x22/0x31 [<ffffffff810029f3>] call_function_single_interrupt+0x13/0x20 <EOI> [<ffffffff8102c9a5>] ? wake_up_process+0x10/0x12 [<ffffffff81008701>] ? default_idle+0x22/0x37 [<ffffffff8100896d>] c1e_idle+0xdf/0xe6 [<ffffffff813f1170>] ? atomic_notifier_call_chain+0x13/0x15 [<ffffffff810012fb>] cpu_idle+0x4b/0x7e [<ffffffff813e8a4e>] start_secondary+0x1ae/0x1b2 Code: 89 e5 41 55 49 89 fd 41 54 45 31 e4 53 31 db 48 83 ec 08 89 df e8 be f8 ff ff 48 98 48 83 3c c5 10 67 7a 81 00 74 1f 49 8b 45 08 <42> 8b 0c 20 0f 32 48 c1 e2 20 25 ff ff bf ff 48 09 d0 48 89 c2 RIP [<ffffffff8132ad57>] op_amd_stop+0x2d/0x8e RSP <ffff880001843f28> CR2: 0000000000000000 ---[ end trace 679ac372d674b757 ]--- Kernel panic - not syncing: Fatal exception in interrupt Pid: 0, comm: swapper Tainted: G D 2.6.34-rc5-oprofile-x86_64-standard-00210-g8c00f06 #16 Call Trace: <IRQ> [<ffffffff813ebd6a>] panic+0x9e/0x10c [<ffffffff810474b0>] ? up+0x34/0x39 [<ffffffff81031ccc>] ? kmsg_dump+0x112/0x12c [<ffffffff813eeff1>] oops_end+0x81/0x8e [<ffffffff8101efee>] no_context+0x1f3/0x202 [<ffffffff8101f1b7>] __bad_area_nosemaphore+0x1ba/0x1e0 [<ffffffff81028d24>] ? enqueue_task_fair+0x16d/0x17a [<ffffffff810264dc>] ? activate_task+0x42/0x53 [<ffffffff8102c967>] ? try_to_wake_up+0x272/0x284 [<ffffffff8101f1eb>] bad_area_nosemaphore+0xe/0x10 [<ffffffff813f0f3f>] do_page_fault+0x1c8/0x37c [<ffffffff81028d24>] ? enqueue_task_fair+0x16d/0x17a [<ffffffff813ee55f>] page_fault+0x1f/0x30 [<ffffffff8102c9a5>] ? wake_up_process+0x10/0x12 [<ffffffff8132ad57>] ? op_amd_stop+0x2d/0x8e [<ffffffff8132ad46>] ? op_amd_stop+0x1c/0x8e [<ffffffff8132a602>] nmi_cpu_stop+0x21/0x23 [<ffffffff810521b3>] generic_smp_call_function_single_interrupt+0xdf/0x11b [<ffffffff8101804f>] smp_call_function_single_interrupt+0x22/0x31 [<ffffffff810029f3>] call_function_single_interrupt+0x13/0x20 <EOI> [<ffffffff8102c9a5>] ? wake_up_process+0x10/0x12 [<ffffffff81008701>] ? default_idle+0x22/0x37 [<ffffffff8100896d>] c1e_idle+0xdf/0xe6 [<ffffffff813f1170>] ? atomic_notifier_call_chain+0x13/0x15 [<ffffffff810012fb>] cpu_idle+0x4b/0x7e [<ffffffff813e8a4e>] start_secondary+0x1ae/0x1b2 ------------[ cut here ]------------ WARNING: at /local/rrichter/.source/linux/arch/x86/kernel/smp.c:118 native_smp_send_reschedule+0x27/0x53() Hardware name: Anaheim Modules linked in: Pid: 0, comm: swapper Tainted: G D 2.6.34-rc5-oprofile-x86_64-standard-00210-g8c00f06 #16 Call Trace: <IRQ> [<ffffffff81017f32>] ? native_smp_send_reschedule+0x27/0x53 [<ffffffff81030ee2>] warn_slowpath_common+0x77/0xa4 [<ffffffff81030f1e>] warn_slowpath_null+0xf/0x11 [<ffffffff81017f32>] native_smp_send_reschedule+0x27/0x53 [<ffffffff8102634b>] resched_task+0x60/0x62 [<ffffffff8102653a>] check_preempt_curr_idle+0x10/0x12 [<ffffffff8102c8ea>] try_to_wake_up+0x1f5/0x284 [<ffffffff8102c986>] default_wake_function+0xd/0xf [<ffffffff810a110d>] pollwake+0x57/0x5a [<ffffffff8102c979>] ? default_wake_function+0x0/0xf [<ffffffff81026be5>] __wake_up_common+0x46/0x75 [<ffffffff81026ed0>] __wake_up+0x38/0x50 [<ffffffff81031694>] printk_tick+0x39/0x3b [<ffffffff8103ac37>] update_process_times+0x3f/0x5c [<ffffffff8104dc63>] tick_periodic+0x5d/0x69 [<ffffffff8104dc90>] tick_handle_periodic+0x21/0x71 [<ffffffff81018fd0>] smp_apic_timer_interrupt+0x82/0x95 [<ffffffff81002853>] apic_timer_interrupt+0x13/0x20 [<ffffffff81030cb5>] ? panic_blink_one_second+0x0/0x7b [<ffffffff813ebdd6>] ? panic+0x10a/0x10c [<ffffffff810474b0>] ? up+0x34/0x39 [<ffffffff81031ccc>] ? kmsg_dump+0x112/0x12c [<ffffffff813eeff1>] ? oops_end+0x81/0x8e [<ffffffff8101efee>] ? no_context+0x1f3/0x202 [<ffffffff8101f1b7>] ? __bad_area_nosemaphore+0x1ba/0x1e0 [<ffffffff81028d24>] ? enqueue_task_fair+0x16d/0x17a [<ffffffff810264dc>] ? activate_task+0x42/0x53 [<ffffffff8102c967>] ? try_to_wake_up+0x272/0x284 [<ffffffff8101f1eb>] ? bad_area_nosemaphore+0xe/0x10 [<ffffffff813f0f3f>] ? do_page_fault+0x1c8/0x37c [<ffffffff81028d24>] ? enqueue_task_fair+0x16d/0x17a [<ffffffff813ee55f>] ? page_fault+0x1f/0x30 [<ffffffff8102c9a5>] ? wake_up_process+0x10/0x12 [<ffffffff8132ad57>] ? op_amd_stop+0x2d/0x8e [<ffffffff8132ad46>] ? op_amd_stop+0x1c/0x8e [<ffffffff8132a602>] ? nmi_cpu_stop+0x21/0x23 [<ffffffff810521b3>] ? generic_smp_call_function_single_interrupt+0xdf/0x11b [<ffffffff8101804f>] ? smp_call_function_single_interrupt+0x22/0x31 [<ffffffff810029f3>] ? call_function_single_interrupt+0x13/0x20 <EOI> [<ffffffff8102c9a5>] ? wake_up_process+0x10/0x12 [<ffffffff81008701>] ? default_idle+0x22/0x37 [<ffffffff8100896d>] ? c1e_idle+0xdf/0xe6 [<ffffffff813f1170>] ? atomic_notifier_call_chain+0x13/0x15 [<ffffffff810012fb>] ? cpu_idle+0x4b/0x7e [<ffffffff813e8a4e>] ? start_secondary+0x1ae/0x1b2 ---[ end trace 679ac372d674b758 ]--- Cc: Andi Kleen <andi@firstfloor.org> Signed-off-by: Robert Richter <robert.richter@amd.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* x86, amd: Check X86_FEATURE_OSVW bit before accessing OSVW MSRsAndreas Herrmann2010-05-261-5/+7
| | | | | | | | | | | | | | commit f01487119dda3d9f58c9729c7361ecc50a61c188 upstream. If host CPU is exposed to a guest the OSVW MSRs are not guaranteed to be present and a GP fault occurs. Thus checking the feature flag is essential. Signed-off-by: Andreas Herrmann <andreas.herrmann3@amd.com> LKML-Reference: <20100427101348.GC4489@alberich.amd.com> Signed-off-by: H. Peter Anvin <hpa@linux.intel.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* x86, cacheinfo: Turn off L3 cache index disable feature in virtualized ↵Frank Arnold2010-05-261-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | environments commit 7f284d3cc96e02468a42e045f77af11e5ff8b095 upstream. When running a quest kernel on xen we get: BUG: unable to handle kernel NULL pointer dereference at 0000000000000038 IP: [<ffffffff8142f2fb>] cpuid4_cache_lookup_regs+0x2ca/0x3df PGD 0 Oops: 0000 [#1] SMP last sysfs file: CPU 0 Modules linked in: Pid: 0, comm: swapper Tainted: G W 2.6.34-rc3 #1 /HVM domU RIP: 0010:[<ffffffff8142f2fb>] [<ffffffff8142f2fb>] cpuid4_cache_lookup_regs+0x 2ca/0x3df RSP: 0018:ffff880002203e08 EFLAGS: 00010046 RAX: 0000000000000000 RBX: 0000000000000003 RCX: 0000000000000060 RDX: 0000000000000000 RSI: 0000000000000040 RDI: 0000000000000000 RBP: ffff880002203ed8 R08: 00000000000017c0 R09: ffff880002203e38 R10: ffff8800023d5d40 R11: ffffffff81a01e28 R12: ffff880187e6f5c0 R13: ffff880002203e34 R14: ffff880002203e58 R15: ffff880002203e68 FS: 0000000000000000(0000) GS:ffff880002200000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000038 CR3: 0000000001a3c000 CR4: 00000000000006f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process swapper (pid: 0, threadinfo ffffffff81a00000, task ffffffff81a44020) Stack: ffffffff810d7ecb ffff880002203e20 ffffffff81059140 ffff880002203e30 <0> ffffffff810d7ec9 0000000002203e40 000000000050d140 ffff880002203e70 <0> 0000000002008140 0000000000000086 ffff880040020140 ffffffff81068b8b Call Trace: <IRQ> [<ffffffff810d7ecb>] ? sync_supers_timer_fn+0x0/0x1c [<ffffffff81059140>] ? mod_timer+0x23/0x25 [<ffffffff810d7ec9>] ? arm_supers_timer+0x34/0x36 [<ffffffff81068b8b>] ? hrtimer_get_next_event+0xa7/0xc3 [<ffffffff81058e85>] ? get_next_timer_interrupt+0x19a/0x20d [<ffffffff8142fa23>] get_cpu_leaves+0x5c/0x232 [<ffffffff8106a7b1>] ? sched_clock_local+0x1c/0x82 [<ffffffff8106a9a0>] ? sched_clock_tick+0x75/0x7a [<ffffffff8107748c>] generic_smp_call_function_single_interrupt+0xae/0xd0 [<ffffffff8101f6ef>] smp_call_function_single_interrupt+0x18/0x27 [<ffffffff8100a773>] call_function_single_interrupt+0x13/0x20 <EOI> [<ffffffff8143c468>] ? notifier_call_chain+0x14/0x63 [<ffffffff810295c6>] ? native_safe_halt+0xc/0xd [<ffffffff810114eb>] ? default_idle+0x36/0x53 [<ffffffff81008c22>] cpu_idle+0xaa/0xe4 [<ffffffff81423a9a>] rest_init+0x7e/0x80 [<ffffffff81b10dd2>] start_kernel+0x40e/0x419 [<ffffffff81b102c8>] x86_64_start_reservations+0xb3/0xb7 [<ffffffff81b103c4>] x86_64_start_kernel+0xf8/0x107 Code: 14 d5 40 ff ae 81 8b 14 02 31 c0 3b 15 47 1c 8b 00 7d 0e 48 8b 05 36 1c 8b 00 48 63 d2 48 8b 04 d0 c7 85 5c ff ff ff 00 00 00 00 <8b> 70 38 48 8d 8d 5c ff ff ff 48 8b 78 10 ba c4 01 00 00 e8 eb RIP [<ffffffff8142f2fb>] cpuid4_cache_lookup_regs+0x2ca/0x3df RSP <ffff880002203e08> CR2: 0000000000000038 ---[ end trace a7919e7f17c0a726 ]--- The L3 cache index disable feature of AMD CPUs has to be disabled if the kernel is running as guest on top of a hypervisor because northbridge devices are not available to the guest. Currently, this fixes a boot crash on top of Xen. In the future this will become an issue on KVM as well. Check if northbridge devices are present and do not enable the feature if there are none. [ hpa: backported to 2.6.34 ] Signed-off-by: Frank Arnold <frank.arnold@amd.com> LKML-Reference: <1271945222-5283-3-git-send-email-bp@amd64.org> Acked-by: Borislav Petkov <borislav.petkov@amd.com> Signed-off-by: H. Peter Anvin <hpa@zytor.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* x86, k8: Fix build error when K8_NB is disabledBorislav Petkov2010-05-261-0/+5
| | | | | | | | | | | | | | commit ade029e2aaacc8965a548b0b0f80c5bee97ffc68 upstream. K8_NB depends on PCI and when the last is disabled (allnoconfig) we fail at the final linking stage due to missing exported num_k8_northbridges. Add a header stub for that. Signed-off-by: Borislav Petkov <borislav.petkov@amd.com> LKML-Reference: <20100503183036.GJ26107@aftab> Signed-off-by: H. Peter Anvin <hpa@zytor.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* x86, k8 nb: Fix boot crash: enable k8_northbridges unconditionally on AMD ↵Borislav Petkov2010-05-123-3/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | systems commit 0e152cd7c16832bd5cadee0c2e41d9959bc9b6f9 upstream. de957628ce7c84764ff41331111036b3ae5bad0f changed setting of the x86_init.iommu.iommu_init function ptr only when GART IOMMU is found. One side effect of it is that num_k8_northbridges is not initialized anymore if not explicitly called. This resulted in uninitialized pointers in <arch/x86/kernel/cpu/intel_cacheinfo.c:amd_calc_l3_indices()>, for example, which uses the num_k8_northbridges thing through node_to_k8_nb_misc(). Fix that through an initcall that runs right after the PCI subsystem and does all the scanning. Then, remove initialization in gart_iommu_init() which is a rootfs_initcall and we're running before that. What is more, since num_k8_northbridges is being used in other places beside GART IOMMU, include it whenever we add AMD CPU support. The previous dependency chain in kconfig contained K8_NB depends on AGP_AMD64|GART_IOMMU which was clearly incorrect. The more natural way in terms of hardware dependency should be AGP_AMD64|GART_IOMMU depends on K8_NB depends on CPU_SUP_AMD && PCI. Make it so Number One! Signed-off-by: Borislav Petkov <borislav.petkov@amd.com> Cc: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp> Cc: Joerg Roedel <joerg.roedel@amd.com> LKML-Reference: <20100312144303.GA29262@aftab> Signed-off-by: Ingo Molnar <mingo@elte.hu> Tested-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* x86: Fix NULL pointer access in irq_force_complete_move() for Xen guestsPrarit Bhargava2010-05-121-0/+3
| | | | | | | | | | | | | | | | | commit bbd391a15d82e14efe9d69ba64cadb855b061dba upstream. Upstream PV guests fail to boot because of a NULL pointer in irq_force_complete_move(). It is possible that xen guests have irq_desc->chip_data = NULL. Test for NULL chip_data pointer before attempting to complete an irq move. Signed-off-by: Prarit Bhargava <prarit@redhat.com> LKML-Reference: <20100427152434.16193.49104.sendpatchset@prarit.bos.redhat.com> Acked-by: Suresh Siddha <suresh.b.siddha@intel.com> Signed-off-by: H. Peter Anvin <hpa@zytor.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* x86: Disable large pages on CPUs with Atom erratum AAE44H. Peter Anvin2010-05-121-0/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | commit 7a0fc404ae663776e96db43879a0fa24fec1fa3a upstream. Atom erratum AAE44/AAF40/AAG38/AAH41: "If software clears the PS (page size) bit in a present PDE (page directory entry), that will cause linear addresses mapped through this PDE to use 4-KByte pages instead of using a large page after old TLB entries are invalidated. Due to this erratum, if a code fetch uses this PDE before the TLB entry for the large page is invalidated then it may fetch from a different physical address than specified by either the old large page translation or the new 4-KByte page translation. This erratum may also cause speculative code fetches from incorrect addresses." [http://download.intel.com/design/processor/specupdt/319536.pdf] Where as commit 211b3d03c7400f48a781977a50104c9d12f4e229 seems to workaround errata AAH41 (mixed 4K TLBs) it reduces the window of opportunity for the bug to occur and does not totally remove it. This patch disables mixed 4K/4MB page tables totally avoiding the page splitting and not tripping this processor issue. This is based on an original patch by Colin King. Originally-by: Colin Ian King <colin.king@canonical.com> Cc: Colin Ian King <colin.king@canonical.com> Cc: Ingo Molnar <mingo@elte.hu> Signed-off-by: H. Peter Anvin <hpa@linux.intel.com> LKML-Reference: <1269271251-19775-1-git-send-email-colin.king@canonical.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* x86-64: Clear a 64-bit FS/GS base on fork if selector is nonzeroH. Peter Anvin2010-05-121-2/+2
| | | | | | | | | | | | | | | | | | commit 7ce5a2b9bb2e92902230e3121d8c3047fab9cb47 upstream. When we do a thread switch, we clear the outgoing FS/GS base if the corresponding selector is nonzero. This is taken by __switch_to() as an entry invariant; it does not verify that it is true on entry. However, copy_thread() doesn't enforce this constraint, which can result in inconsistent results after fork(). Make copy_thread() match the behavior of __switch_to(). Reported-and-tested-by: Samuel Thibault <samuel.thibault@inria.fr> Signed-off-by: H. Peter Anvin <hpa@zytor.com> LKML-Reference: <4BD1E061.8030605@zytor.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* powernow-k8: Fix frequency reportingMark Langsdorf2010-05-121-1/+2
| | | | | | | | | | | | | | | commit b810e94c9d8e3fff6741b66cd5a6f099a7887871 upstream. With F10, model 10, all valid frequencies are in the ACPI _PST table. Signed-off-by: Mark Langsdorf <mark.langsdorf@amd.com> LKML-Reference: <1270065406-1814-6-git-send-email-bp@amd64.org> Signed-off-by: Borislav Petkov <borislav.petkov@amd.com> Reviewed-by: Thomas Renninger <trenn@suse.de> Signed-off-by: H. Peter Anvin <hpa@zytor.com> Signed-off-by: Ingo Molnar <mingo@elte.hu> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* x86/gart: Disable GART explicitly before initializationJoerg Roedel2010-04-262-1/+17
| | | | | | | | | | | | | | commit 4b83873d3da0704987cb116833818ed96214ee29 upstream. If we boot into a crash-kernel the gart might still be enabled and its caches might be dirty. This can result in undefined behavior later. Fix it by explicitly disabling the gart hardware before initialization and flushing the caches after enablement. Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* KVM: x86: Fix TSS size check for 16-bit tasksJan Kiszka2010-04-261-1/+5
| | | | | | | | | | | | (Cherry-picked from commit e8861cfe2c75bdce36655b64d7ce02c2b31b604d) A 16-bit TSS is only 44 bytes long. So make sure to test for the correct size on task switch. Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com> Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* KVM: fix the handling of dirty bitmaps to avoid overflowsTakuya Yoshikawa2010-04-261-2/+2
| | | | | | | | | | | | | | | | | (Cherry-picked from commit 87bf6e7de1134f48681fd2ce4b7c1ec45458cb6d) Int is not long enough to store the size of a dirty bitmap. This patch fixes this problem with the introduction of a wrapper function to calculate the sizes of dirty bitmaps. Note: in mark_page_dirty(), we have to consider the fact that __set_bit() takes the offset as int, not long. Signed-off-by: Takuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* KVM: MMU: fix kvm_mmu_zap_page() and its calling pathXiao Guangrong2010-04-261-4/+7
| | | | | | | | | | | | | | | | (Cherry-picked from commit 77662e0028c7c63e34257fda03ff9625c59d939d) This patch fix: - calculate zapped page number properly in mmu_zap_unsync_children() - calculate freeed page number properly kvm_mmu_change_mmu_pages() - if zapped children page it shoud restart hlist walking KVM-Stable-Tag. Signed-off-by: Xiao Guangrong <xiaoguangrong@cn.fujitsu.com> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* KVM: VMX: Save/restore rflags.vm correctly in real modeAvi Kivity2010-04-261-9/+15
| | | | | | | | | | | | | | | | | | | | | | (Cherry-picked from commit 78ac8b47c566dd6177a3b9b291b756ccb70670b7) Currently we set eflags.vm unconditionally when entering real mode emulation through virtual-8086 mode, and clear it unconditionally when we enter protected mode. The means that the following sequence KVM_SET_REGS (rflags.vm=1) KVM_SET_SREGS (cr0.pe=1) Ends up with rflags.vm clear due to KVM_SET_SREGS triggering enter_pmode(). Fix by shadowing rflags.vm (and rflags.iopl) correctly while in real mode: reads and writes to those bits access a shadow register instead of the actual register. Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* KVM: allow bit 10 to be cleared in MSR_IA32_MC4_CTLAndre Przywara2010-04-261-2/+6
| | | | | | | | | | | | | | | | | | (Cherry-picked from commit 114be429c8cd44e57f312af2bbd6734e5a185b0d) There is a quirk for AMD K8 CPUs in many Linux kernels (see arch/x86/kernel/cpu/mcheck/mce.c:__mcheck_cpu_apply_quirks()) that clears bit 10 in that MCE related MSR. KVM can only cope with all zeros or all ones, so it will inject a #GP into the guest, which will let it panic. So lets add a quirk to the quirk and ignore this single cleared bit. This fixes -cpu kvm64 on all machines and -cpu host on K8 machines with some guest Linux kernels. Signed-off-by: Andre Przywara <andre.przywara@amd.com> Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* KVM: Don't spam kernel log when injecting exceptions due to bad cr writesAvi Kivity2010-04-261-27/+0
| | | | | | | | | | (Cherry-picked from commit d6a23895aa82353788a1cc5a1d9a1c963465463e) These are guest-triggerable. Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* KVM: SVM: Fix memory leaks that happen when svm_create_vcpu() failsTakuya Yoshikawa2010-04-261-10/+15
| | | | | | | | | | | | (Cherry-picked from commit b7af40433870aa0636932ad39b0c48a0cb319057) svm_create_vcpu() does not free the pages allocated during the creation when it fails to complete the allocations. This patch fixes it. Signed-off-by: Takuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp> Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* KVM: VMX: Update instruction length on intercepted BPJan Kiszka2010-04-261-0/+13
| | | | | | | | | | | | | | | (Cherry-picked from commit c573cd22939e54fc1b8e672054a505048987a7cb) We intercept #BP while in guest debugging mode. As VM exits due to intercepted exceptions do not necessarily come with valid idt_vectoring, we have to update event_exit_inst_len explicitly in such cases. At least in the absence of migration, this ensures that re-injections of #BP will find and use the correct instruction length. Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com> Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* perf_events, x86: Implement Intel Westmere/Nehalem-EX supportPeter Zijlstra2010-04-261-1/+103
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | original patch commit ids: 452a339a976e7f782c786eb3f73080401e2fa3a6 and 134fbadf028a5977a1b06b0253d3ee33e6f0c642 perf_events, x86: Implement Intel Westmere support The new Intel documentation includes Westmere arch specific event maps that are significantly different from the Nehalem ones. Add support for this generation. Found the CPUID model numbers on wikipedia. Also ammend some Nehalem constraints, spotted those when looking for the differences between Nehalem and Westmere. Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Arjan van de Ven <arjan@linux.intel.com> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: Stephane Eranian <eranian@google.com> LKML-Reference: <20100127221122.151865645@chello.nl> Signed-off-by: Ingo Molnar <mingo@elte.hu> perf, x86: Enable Nehalem-EX support According to Intel Software Devel Manual Volume 3B, the Nehalem-EX PMU is just like regular Nehalem (except for the uncore support, which is completely different). Signed-off-by: Vince Weaver <vweaver1@eecs.utk.edu> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Paul Mackerras <paulus@samba.org> Cc: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: Arjan van de Ven <arjan@linux.intel.com> Cc: Lin Ming <ming.m.lin@intel.com> LKML-Reference: <alpine.DEB.2.00.1004060956580.1417@cl320.eecs.utk.edu> Signed-off-by: Ingo Molnar <mingo@elte.hu> Cc: Youquan Song <youquan.song@linux.intel.com>
* x86/PCI: irq and pci_ids patch for Intel Cougar Point DeviceIDsSeth Heasley2010-04-261-0/+2
| | | | | | | | | | | | commit 93da6202264ce1256b04db8008a43882ae62d060 upstream. This patch adds the Intel Cougar Point (PCH) LPC and SMBus Controller DeviceIDs. Signed-off-by: Seth Heasley <seth.heasley@intel.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org> Cc: maximilian attems <max@stro.at> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* x86-64, rwsem: Avoid store forwarding hazard in __downgrade_writeAvi Kivity2010-04-261-20/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | commit 0d1622d7f526311d87d7da2ee7dd14b73e45d3fc upstream. The Intel Architecture Optimization Reference Manual states that a short load that follows a long store to the same object will suffer a store forwading penalty, particularly if the two accesses use different addresses. Trivially, a long load that follows a short store will also suffer a penalty. __downgrade_write() in rwsem incurs both penalties: the increment operation will not be able to reuse a recently-loaded rwsem value, and its result will not be reused by any recently-following rwsem operation. A comment in the code states that this is because 64-bit immediates are special and expensive; but while they are slightly special (only a single instruction allows them), they aren't expensive: a test shows that two loops, one loading a 32-bit immediate and one loading a 64-bit immediate, both take 1.5 cycles per iteration. Fix this by changing __downgrade_write to use the same add instruction on i386 and on x86_64, so that it uses the same operand size as all the other rwsem functions. Signed-off-by: Avi Kivity <avi@redhat.com> LKML-Reference: <1266049992-17419-1-git-send-email-avi@redhat.com> Signed-off-by: H. Peter Anvin <hpa@zytor.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* x86-64: support native xadd rwsem implementationLinus Torvalds2010-04-263-1/+83
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | commit bafaecd11df15ad5b1e598adc7736afcd38ee13d upstream. This one is much faster than the spinlock based fallback rwsem code, with certain artifical benchmarks having shown 300%+ improvement on threaded page faults etc. Again, note the 32767-thread limit here. So this really does need that whole "make rwsem_count_t be 64-bit and fix the BIAS values to match" extension on top of it, but that is conceptually a totally independent issue. NOT TESTED! The original patch that this all was based on were tested by KAMEZAWA Hiroyuki, but maybe I screwed up something when I created the cleaned-up series, so caveat emptor.. Also note that it _may_ be a good idea to mark some more registers clobbered on x86-64 in the inline asms instead of saving/restoring them. They are inline functions, but they are only used in places where there are not a lot of live registers _anyway_, so doing for example the clobbers of %r8-%r11 in the asm wouldn't make the fast-path code any worse, and would make the slow-path code smaller. (Not that the slow-path really matters to that degree. Saving a few unnecessary registers is the _least_ of our problems when we hit the slow path. The instruction/cycle counting really only matters in the fast path). Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> LKML-Reference: <alpine.LFD.2.00.1001121810410.17145@localhost.localdomain> Signed-off-by: H. Peter Anvin <hpa@zytor.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* x86-64, rwsem: 64-bit xadd rwsem implementationH. Peter Anvin2010-04-261-13/+40
| | | | | | | | | | | | | | | | | | | | | | | | | commit 1838ef1d782f7527e6defe87e180598622d2d071 upstream. For x86-64, 32767 threads really is not enough. Change rwsem_count_t to a signed long, so that it is 64 bits on x86-64. This required the following changes to the assembly code: a) %z0 doesn't work on all versions of gcc! At least gcc 4.4.2 as shipped with Fedora 12 emits "ll" not "q" for 64 bits, even for integer operands. Newer gccs apparently do this correctly, but avoid this problem by using the _ASM_ macros instead of %z. b) 64 bits immediates are only allowed in "movq $imm,%reg" constructs... no others. Change some of the constraints to "e", and fix the one case where we would have had to use an invalid immediate -- in that case, we only care about the upper half anyway, so just access the upper half. Signed-off-by: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> LKML-Reference: <tip-bafaecd11df15ad5b1e598adc7736afcd38ee13d@git.kernel.org> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* x86: clean up rwsem type systemLinus Torvalds2010-04-261-10/+15
| | | | | | | | | | | | | | | | | | | | | | | | commit 5d0b7235d83eefdafda300656e97d368afcafc9a upstream. The fast version of the rwsems (the code that uses xadd) has traditionally only worked on x86-32, and as a result it mixes different kinds of types wildly - they just all happen to be 32-bit. We have "long", we have "__s32", and we have "int". To make it work on x86-64, the types suddenly matter a lot more. It can be either a 32-bit or 64-bit signed type, and both work (with the caveat that a 32-bit counter will only have 15 bits of effective write counters, so it's limited to 32767 users). But whatever type you choose, it needs to be used consistently. This makes a new 'rwsem_counter_t', that is a 32-bit signed type. For a 64-bit type, you'd need to also update the BIAS values. Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> LKML-Reference: <alpine.LFD.2.00.1001121755220.17145@localhost.localdomain> Signed-off-by: H. Peter Anvin <hpa@zytor.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* x86-32: clean up rwsem inline asm statementsLinus Torvalds2010-04-261-15/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | commit 59c33fa7791e9948ba467c2b83e307a0d087ab49 upstream. This makes gcc use the right register names and instruction operand sizes automatically for the rwsem inline asm statements. So instead of using "(%%eax)" to specify the memory address that is the semaphore, we use "(%1)" or similar. And instead of forcing the operation to always be 32-bit, we use "%z0", taking the size from the actual semaphore data structure itself. This doesn't actually matter on x86-32, but if we want to use the same inline asm for x86-64, we'll need to have the compiler generate the proper 64-bit names for the registers (%rax instead of %eax), and if we want to use a 64-bit counter too (in order to avoid the 15-bit limit on the write counter that limits concurrent users to 32767 threads), we'll need to be able to generate instructions with "q" accesses rather than "l". Since this header currently isn't enabled on x86-64, none of that matters, but we do want to use the xadd version of the semaphores rather than have to take spinlocks to do a rwsem. The mm->mmap_sem can be heavily contended when you have lots of threads all taking page faults, and the fallback rwsem code that uses a spinlock performs abysmally badly in that case. [ hpa: modified the patch to skip size suffixes entirely when they are redundant due to register operands. ] Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> LKML-Reference: <alpine.LFD.2.00.1001121613560.17145@localhost.localdomain> Signed-off-by: H. Peter Anvin <hpa@zytor.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
* x86, cacheinfo: Enable L3 CID only on AMDBorislav Petkov2010-04-261-88/+98
| | | | | | | | | | | | | | | | | | | | | | commit cb19060abfdecac0d1eb2d2f0e7d6b7a3f8bc4f4 upstream. Final stage linking can fail with arch/x86/built-in.o: In function `store_cache_disable': intel_cacheinfo.c:(.text+0xc509): undefined reference to `amd_get_nb_id' arch/x86/built-in.o: In function `show_cache_disable': intel_cacheinfo.c:(.text+0xc7d3): undefined reference to `amd_get_nb_id' when CONFIG_CPU_SUP_AMD is not enabled because the amd_get_nb_id helper is defined in AMD-specific code but also used in generic code (intel_cacheinfo.c). Reorganize the L3 cache index disable code under CONFIG_CPU_SUP_AMD since it is AMD-only anyway. Signed-off-by: Borislav Petkov <borislav.petkov@amd.com> LKML-Reference: <20100218184210.GF20473@aftab> Signed-off-by: H. Peter Anvin <hpa@zytor.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
generated by cgit (git 2.34.1) at 2025-06-14 05:44:27 +0000