<feed xmlns='http://www.w3.org/2005/Atom'>
<title>libpinyin.git/src/storage/ngram.cpp, branch tkrzw</title>
<subtitle>mirror of libpinyin.</subtitle>
<link rel='alternate' type='text/html' href='https://fedorapeople.org/cgit/pwu/public_git/libpinyin.git/'/>
<entry>
<title>Fix libpinyin crash on ARMv7</title>
<updated>2022-11-29T08:16:21+00:00</updated>
<author>
<name>Matias Larsson</name>
<email>matias.larsson@matthews.se</email>
</author>
<published>2022-11-24T17:22:55+00:00</published>
<link rel='alternate' type='text/html' href='https://fedorapeople.org/cgit/pwu/public_git/libpinyin.git/commit/?id=8bcea4710bd328ef74dc852d9fdffb3c47ed8abe'/>
<id>8bcea4710bd328ef74dc852d9fdffb3c47ed8abe</id>
<content type='text'>
Fix the alignment trap in get_unigram_frequency(). Fix also other
places where this same trap could happen (depending on compiler
and surrounding code).

The trap happened when the ARM GCC generated a SIMD instruction
(specifically VLDR) to load 32 bits in a single instruction, and
when the memory address was not aligned to 32 bits. VLDR traps if
the address is not aligned.

GCC generated the instruction because of the cast to uint32 from
the address. The fix is to allocate a uint32 variable in stack and
use memcpy to copy the data to that variable. This way we ensure
that appropriate instructions are generated.

**Links**

About the issue with GCC:
https://trust-in-soft.com/blog/2020/04/06/gcc-always-assumes-aligned-pointer-accesses/

How Linux does it:
https://elixir.bootlin.com/linux/v5.10.155/source/include/linux/unaligned/memmove.h#L13

ARM documentation:
https://documentation-service.arm.com/static/5f8dc043f86e16515cdbbc92?token=
    See 'A3.2.1 Unaligned data access'
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Fix the alignment trap in get_unigram_frequency(). Fix also other
places where this same trap could happen (depending on compiler
and surrounding code).

The trap happened when the ARM GCC generated a SIMD instruction
(specifically VLDR) to load 32 bits in a single instruction, and
when the memory address was not aligned to 32 bits. VLDR traps if
the address is not aligned.

GCC generated the instruction because of the cast to uint32 from
the address. The fix is to allocate a uint32 variable in stack and
use memcpy to copy the data to that variable. This way we ensure
that appropriate instructions are generated.

**Links**

About the issue with GCC:
https://trust-in-soft.com/blog/2020/04/06/gcc-always-assumes-aligned-pointer-accesses/

How Linux does it:
https://elixir.bootlin.com/linux/v5.10.155/source/include/linux/unaligned/memmove.h#L13

ARM documentation:
https://documentation-service.arm.com/static/5f8dc043f86e16515cdbbc92?token=
    See 'A3.2.1 Unaligned data access'
</pre>
</div>
</content>
</entry>
<entry>
<title>Use abort function instead of assert macro</title>
<updated>2022-06-06T09:32:00+00:00</updated>
<author>
<name>Peng Wu</name>
<email>alexepico@gmail.com</email>
</author>
<published>2022-06-06T09:32:00+00:00</published>
<link rel='alternate' type='text/html' href='https://fedorapeople.org/cgit/pwu/public_git/libpinyin.git/commit/?id=2323c59d337ca56c585a8b58b855e84938ecfc27'/>
<id>2323c59d337ca56c585a8b58b855e84938ecfc27</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Use check_result macro in src directory</title>
<updated>2022-06-01T04:06:24+00:00</updated>
<author>
<name>Peng Wu</name>
<email>alexepico@gmail.com</email>
</author>
<published>2022-06-01T03:47:23+00:00</published>
<link rel='alternate' type='text/html' href='https://fedorapeople.org/cgit/pwu/public_git/libpinyin.git/commit/?id=706188e5080f29665ecd41791df611f4e3a6e248'/>
<id>706188e5080f29665ecd41791df611f4e3a6e248</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>fixes merge_single_gram function</title>
<updated>2020-08-21T07:55:45+00:00</updated>
<author>
<name>Peng Wu</name>
<email>alexepico@gmail.com</email>
</author>
<published>2020-08-21T07:55:45+00:00</published>
<link rel='alternate' type='text/html' href='https://fedorapeople.org/cgit/pwu/public_git/libpinyin.git/commit/?id=89972e4d34331fe199633b10408f0656f7361e32'/>
<id>89972e4d34331fe199633b10408f0656f7361e32</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Copy the data when merge into single ngram.</title>
<updated>2020-08-18T02:09:56+00:00</updated>
<author>
<name>Weng Xuetian</name>
<email>xweng@google.com</email>
</author>
<published>2020-08-17T07:41:33+00:00</published>
<link rel='alternate' type='text/html' href='https://fedorapeople.org/cgit/pwu/public_git/libpinyin.git/commit/?id=15adb1dbe553bc38c5a2a9032dbf6963cc859772'/>
<id>15adb1dbe553bc38c5a2a9032dbf6963cc859772</id>
<content type='text'>
system and user single ngram might be invalidated later.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
system and user single ngram might be invalidated later.
</pre>
</div>
</content>
</entry>
<entry>
<title>update code license</title>
<updated>2016-10-31T05:56:38+00:00</updated>
<author>
<name>Peng Wu</name>
<email>alexepico@gmail.com</email>
</author>
<published>2016-10-31T05:56:38+00:00</published>
<link rel='alternate' type='text/html' href='https://fedorapeople.org/cgit/pwu/public_git/libpinyin.git/commit/?id=62cacd1dc8784f7d3e99503d0e8e66010597eb05'/>
<id>62cacd1dc8784f7d3e99503d0e8e66010597eb05</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>add copy parameter to single gram constructor</title>
<updated>2015-04-21T03:13:50+00:00</updated>
<author>
<name>Peng Wu</name>
<email>alexepico@gmail.com</email>
</author>
<published>2015-04-21T03:13:50+00:00</published>
<link rel='alternate' type='text/html' href='https://fedorapeople.org/cgit/pwu/public_git/libpinyin.git/commit/?id=fb3a0fa827098b9c2f8885e1e55e761deccdac1d'/>
<id>fb3a0fa827098b9c2f8885e1e55e761deccdac1d</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>re-factor Bigram</title>
<updated>2015-04-09T08:02:05+00:00</updated>
<author>
<name>Peng Wu</name>
<email>alexepico@gmail.com</email>
</author>
<published>2015-04-09T08:02:05+00:00</published>
<link rel='alternate' type='text/html' href='https://fedorapeople.org/cgit/pwu/public_git/libpinyin.git/commit/?id=941d6c364906a932b4f2874177962e50188e38de'/>
<id>941d6c364906a932b4f2874177962e50188e38de</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>fixes berkeley db calls</title>
<updated>2013-06-18T03:04:29+00:00</updated>
<author>
<name>Peng Wu</name>
<email>alexepico@gmail.com</email>
</author>
<published>2013-06-18T02:53:04+00:00</published>
<link rel='alternate' type='text/html' href='https://fedorapeople.org/cgit/pwu/public_git/libpinyin.git/commit/?id=17221a12d28de70bf2d0071f2359a4b13668dfc5'/>
<id>17221a12d28de70bf2d0071f2359a4b13668dfc5</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>write Bigram::mask_out</title>
<updated>2012-10-30T09:33:08+00:00</updated>
<author>
<name>Peng Wu</name>
<email>alexepico@gmail.com</email>
</author>
<published>2012-10-30T09:33:08+00:00</published>
<link rel='alternate' type='text/html' href='https://fedorapeople.org/cgit/pwu/public_git/libpinyin.git/commit/?id=c315a6e31dcbd42d8151e250bed01edcf00932a4'/>
<id>c315a6e31dcbd42d8151e250bed01edcf00932a4</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
</feed>
