glusterfs.git/libglusterfs/src/gfdb/gfdb_sqlite3_helper.c, branch v3.11.0

core: assorted typos and spelling mistakes reported by Debian lintian

2016-05-18T09:21:42+00:00

Also missing bang (!) in #!/bin/bash in shell scripts.

Change-Id: I567a4be8f0f31f6285550f243fe802895f6bc43b
BUG: 1336793
Reported-by: Patrick Matthäi 
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/14398
NetBSD-regression: NetBSD Build System 
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System 
Reviewed-by: Kotresh HR 
CentOS-regression: Gluster Build System

ctr/sql: Providing for vol set for sqlcachesize and sqlWALsize and skip recording path

2015-12-22T15:14:10+00:00

1. Providing vol set option for cache size and wal autocheck point
   so that performance can be tuned.
2. Removed recording of file path in the db. Trimming database columns.
   Path need not be stored in the db, as PARGFID, GFID, Basename is suffice
   to derive the path during migration.

Change-Id: I2cb590451a6d244bc91fe66c6dbffe2c2059dfb8
BUG: 1293034
Signed-off-by: Joseph Fernandes 
Reviewed-on: http://review.gluster.org/12972
Reviewed-by: N Balachandran 
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

tier/ctr: Providing option to record or ignore metadata heat

2015-11-15T21:47:48+00:00

Currently we heat up a file for both data and metadata write.
Here we provide a ctr xlator option called "ctr-record-metadata-heat"
were the admin can decide on recording metadata heat i.e heatup a
file on metadata writes or not.
Metadata data operation are
a. setattr: explicit changing of atime/mtime using utimes,
            changing of posix permissions of the file
b. rename: Renaming a file,
c. unlink, link: adding or deleting hardlinks
d. xattrs: setting or removal of xattrs.

NOTE: atime, mtime and ctime change through writev, readv, truncate, mknod
and create will not be considered here as these fops are data and primary
metadata fops.

Defaultly "ctr-record-metadata-heat" is off. Admin can
switch it on using gluster volume set command.

Change-Id: I91157509255dd5cb429cda2b6d4f64582e155e7b
BUG: 1279166
Signed-off-by: Joseph Fernandes 
Reviewed-on: http://review.gluster.org/12540
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

tier/libgfdb: Extending log level flexibity in libgfdb

2015-11-11T14:21:02+00:00

Extending log level flexibity to relevant fops and operations
This is an extension to
http://review.gluster.org/#/c/12491/

Change-Id: I33b2f7732f89f52569fb99baa692c7e3bb4c7ab1
BUG: 1277352
Signed-off-by: Joseph Fernandes 
Reviewed-on: http://review.gluster.org/12567
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

tier/ctr: Ignore CTR Lookup heal insert errors

2015-11-07T01:14:03+00:00

CTR doesnt read from the DB, so to make sure that file records are
created it does a heal during a lookup. It remembers the decision in
the inode context cache and retrys periodically. When the volume is
restarted it looses all the inode cache from the previous time and CTR
lookup heals tries the heal again, but this time it finds that the records
are already there from sql and logs this error, and remembers this until the
volume is restarted or inode is flushed out of inode cache of the brick.

Solution: the log levels should be reduced to trace for this case and
customers need not see this.

Change-Id: I67b568fb6904f8597e2c6d32894a247c4f500b94
BUG: 1277352
Signed-off-by: Joseph Fernandes 
Reviewed-on: http://review.gluster.org/12491
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

tier/libgfdb: Replacing ASCII query file with binary

2015-11-06T18:30:45+00:00

Earlier, when the database was queried we used to save
all the queried records in an ASCII format in the query file.
This caused issues like filename having ASCII delimiter and used
to take a lot of space. The tier.c file also had a lot of parsing code.

Here we changed the format of the query file to binary.
All the logic of serialization and formating of query record is done
by libgfdb. Libgfdb provides API,
gfdb_write_query_record() and gfdb_read_query_record(),
which the user i.e tier migrator and CTR xlator can use to
write to and read from query file.
With this binary format we save on disk space i.e reduce to 50% atleast
as we are saving GFID's in binary format 16 bytes and not the string format
which takes 36 bytes + We are not saving path of the file + we are also saving on
ASCII delimiters.

The on disk format of query record is as follows,

+---------------------------------------------------------------------------+
| Length of serialized query record |       Serialized Query Record         |
+---------------------------------------------------------------------------+
             4 bytes                     Length of serialized query record
                                                      |
                                                      |
     -------------------------------------------------|
     |
     |
     V
   Serialized Query Record Format:
   +---------------------------------------------------------------------------+
   | GFID |  Link count   |    |.....                      | FOOTER |
   +---------------------------------------------------------------------------+
     16 B        4 B         Link Length                                  4 B
                                |                                          |
                                |                                          |
   -----------------------------|                                          |
   |                                                                       |
   |                                                                       |
   V                                                                       |
   Each  will be serialized as                                  |
   +-----------------------------------------------+                       |
   | PGID | BASE_NAME_LENGTH |      BASE_NAME      |                       |
   +-----------------------------------------------+                       |
     16 B       4 B             BASE_NAME_LENGTH                           |
                                                                           |
                                                                           |
   ------------------------------------------------------------------------|
   |
   |
   V
   FOOTER is a magic number 0xBAADF00D indicating the end of the record.
   This also serves as a serialized schema validator.

Change-Id: I9db7416fd421e118dd44eafab8b535caafe50d5a
BUG: 1272207
Signed-off-by: Joseph Fernandes 
Reviewed-on: http://review.gluster.org/12354
Reviewed-by: N Balachandran 
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

tier/ctr: Solving DB Lock issue due to write contention from db connections

2015-09-08T12:13:00+00:00

Problem: The DB on the brick is been accessed by CTR, for write and
tier migrator, for read and write. The write from tier migrator is reseting
the heat counters after a cycle. Since we are using sqlite, two connections
trying to write would cause a db lock contention. As a result CTR used to fail
to update the db.

Solution: Using the same db connection of CTR for reseting the heat counters.
1) Introducted a new IPC FOP for CTR
2) After the query do a ipc syncop to the underlying client xlator associated
   to the brick.
3) CTR in brick will catch the IPC FOP and cleat the heat counters.

Change-Id: I53306bfc08dcdba479deb4ccc154896521336150
BUG: 1260730
Signed-off-by: Joseph Fernandes 
Reviewed-on: http://review.gluster.org/12031
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

all: reduce "inline" usage

2015-09-01T11:55:15+00:00

There are three kinds of inline functions: plain inline, extern inline,
and static inline.  All three have been removed from .c files, except
those in "contrib" which aren't our problem.  Inlines in .h files, which
are overwhelmingly "static inline" already, have generally been left
alone.  Over time we should be able to "lower" these into .c files, but
that has to be done in a case-by-case fashion requiring more manual
effort.  This part was easy to do automatically without (as far as I can
tell) any ill effect.

In the process, several pieces of dead code were flagged by the
compiler, and were removed.

Change-Id: I56a5e614735c9e0a6ee420dab949eac22e25c155
BUG: 1245331
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/11769
Tested-by: NetBSD Build System 
Reviewed-by: Dan Lambright 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Niels de Vos 
Reviewed-by: Raghavendra Bhat 
Reviewed-by: Venky Shankar

libgfdb/ctr: Fixing rename or update of link

2015-07-09T19:11:25+00:00

Fixing rename or updating of link in
1) libgfdb : handling return from updatelink
2) ctr xlator: correcting freeing of oldbase name in
               hardlink list in the update link

Change-Id: I1789c492b81876123cd456051a9ae495e876ad46
BUG: 1241054
Signed-off-by: Joseph Fernandes 
Reviewed-on: http://review.gluster.org/11584
Tested-by: Joseph Fernandes
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright 
Tested-by: Gluster Build System

tier/ctr: Ignore creation of T file and Ctr Lookup heal improvememnts

2015-06-26T23:05:20+00:00

1) Ignore creation of T file in ctr_mknod
2) Ignore lookup for T file in ctr_lookup
3) Ctr_lookup:
    a. If the gfid and pgfid in empty dont record
    b. Decreased log level for multiple heal attempts
    c. Inode/File heal happens after an expiry period, which is configurable.
    d. Hardlink heal happens after an expiry period, which is configurable.

Change-Id: Id8eb5092e78beaec22d05f5283645081619e2452
BUG: 1235269
Signed-off-by: Joseph Fernandes 
Reviewed-on: http://review.gluster.org/11334
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright