1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
|
<?xml version='1.0'?>
<!DOCTYPE section PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
]>
<section id="futexcontentionsect">
<title>Identifying Contended User-Space Locks</title>
<indexterm>
<primary>script examples</primary>
<secondary>identifying contended user-space locks (i.e. futex contentions)</secondary>
</indexterm>
<indexterm>
<primary>examples of SystemTap scripts</primary>
<secondary>identifying contended user-space locks (i.e. futex contentions)</secondary>
</indexterm>
<indexterm>
<primary>identifying contended user-space locks (i.e. futex contentions)</primary>
<secondary>examples of SystemTap scripts</secondary>
</indexterm>
<indexterm>
<primary>contended user-space locks (futex contentions), identifying</primary>
<secondary>examples of SystemTap scripts</secondary>
</indexterm>
<remark>
WAR STORY: Futex contention http://sourceware.org/systemtap/wiki/WSFutexContention?highlight=((WarStories))
</remark>
<remark>
no script in examples
</remark>
<para>This section describes how to identify contended user-space locks throughout the system within a specific time period. The ability to identify contended user-space locks can help you investigate hangs that you suspect may be caused by <command>futex</command> contentions.</para>
<indexterm>
<primary>futex contentions, identifying</primary>
<secondary>examples of SystemTap scripts</secondary>
</indexterm>
<indexterm>
<primary>futex contention, definition</primary>
<secondary>examples of SystemTap scripts</secondary>
</indexterm>
<indexterm>
<primary>script examples</primary>
<secondary>process deadlocks (arising from futex contentions)</secondary>
</indexterm>
<indexterm>
<primary>examples of SystemTap scripts</primary>
<secondary>process deadlocks (arising from futex contentions)</secondary>
</indexterm>
<indexterm>
<primary>process deadlocks (arising from futex contentions)</primary>
<secondary>examples of SystemTap scripts</secondary>
</indexterm>
<para>Simply put, a <command>futex</command> contention occurs when multiple processes are trying to access the same region of memory. In some cases, this can result in a deadlock between the processes in contention, thereby appearing as an application hang.</para>
<!-- next 2 indexterms for futex system call -->
<indexterm>
<primary>script examples</primary>
<secondary>futex system call</secondary>
</indexterm>
<indexterm>
<primary>examples of SystemTap scripts</primary>
<secondary>futex system call</secondary>
</indexterm>
<indexterm>
<primary>futex system call</primary>
<secondary>examples of SystemTap scripts</secondary>
</indexterm>
<para>To do this, <xref linkend="futexcontention"/> probes the <command>futex</command> system call.</para>
<formalpara id="futexcontention">
<title>futexes.stp</title>
<para>
<programlisting>
<xi:include parse="text" href="../testsuite/systemtap.examples/process/futexes.stp" xmlns:xi="http://www.w3.org/2001/XInclude" />
</programlisting>
</para>
</formalpara>
<para><xref linkend="futexcontention"/> needs to be manually stopped; upon exit, it prints the following information:</para>
<itemizedlist>
<listitem><para>Name and ID of the process responsible for a contention</para></listitem>
<listitem><para>The region of memory it contested</para></listitem>
<listitem><para>How many times the region of memory was contended</para></listitem>
<listitem><para>Average time of contention throughout the probe</para></listitem>
</itemizedlist>
<para><xref linkend="futexcontentionoutput"/> contains an excerpt from the output of <xref linkend="futexcontention"/> upon exiting the script (after approximately 20 seconds).</para>
<example id="futexcontentionoutput">
<title><xref linkend="futexcontention"/> Sample Output</title>
<screen>
[...]
automount[2825] lock 0x00bc7784 contended 18 times, 999931 avg us
synergyc[3686] lock 0x0861e96c contended 192 times, 101991 avg us
synergyc[3758] lock 0x08d98744 contended 192 times, 101990 avg us
synergyc[3938] lock 0x0982a8b4 contended 192 times, 101997 avg us
[...]
</screen>
</example>
<!-- global thread_thislock # short
global thread_blocktime #
global FUTEX_WAIT = 0, FUTEX_WAKE = 1
global lock_waits # long-lived stats on (tid,lock) blockage elapsed time
global process_names # long-lived pid-to-execname mapping
probe syscall.futex {
if (op != FUTEX_WAIT) next # we don't care about originators of WAKE events
t = tid ()
process_names[pid()] = execname()
thread_thislock[t] = $uaddr
thread_blocktime[t] = gettimeofday_us()
}
probe syscall.futex.return {
t = tid()
ts = thread_blocktime[t]
if (ts) {
elapsed = gettimeofday_us() - ts
lock_waits[pid(), thread_thislock[t]] <<< elapsed
delete thread_blocktime[t]
delete thread_thislock[t]
}
}
probe end {
foreach ([pid+, lock] in lock_waits)
printf ("%s[%d] lock %p contended %d times, %d avg us\n",
process_names[pid], pid, lock, @count(lock_waits[pid,lock]), @avg(lock_waits[pid,lock]))
} -->
</section>
|
