glusterfs.git/api/src/glfs-resolve.c, branch v3.6.0beta2

libgfapi: Fixed an issue with healing files during glfs_resolve

2014-07-16T11:48:20+00:00

          While resolving any path during the first lookup, libgfapi
          should generate and send gfid as well along with the new inode
          created to the syncop_lookup(..) so that POSIX translator
          can heal the files with missing gfid using the new gfid passed.

          This wasn't happening correctly in the current "glfs_resolve_component(..)"
          implementation. Fixed the same.

          Also have added the changes from http://review.gluster.org/5337 in
          libgfapi, which is a fix to unlink the inode on revalidate if entry not found.

          In addition to the above, have cleaned up a redundant gfapi log mesage.

Change-Id: I0757dda782d16ba6bdbe7ebdbde9c43381229b0a
BUG: 1116854
Signed-off-by: Soumya Koduri 
Reviewed-on: http://review.gluster.org/7976
Tested-by: Gluster Build System 
Reviewed-by: Poornima G 
Reviewed-by: Raghavendra G 
Reviewed-by: Vijay Bellur

libgfapi: succeed lookup of "/.."

2014-07-10T04:14:54+00:00

For the root dir, ".." should resolve to itself.

i.e. when
        glfs_h_lookupat (fs, NULL, "/..", &stat)
(or)
        glfs_h_lookupat (fs, root, "..", &stat)

is performed, stat must contain root dir's information.

Change-Id: I1c92091cdc4ff00e6b17e5fa349009c6dfc441c1
BUG: 1114814
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/8207
Tested-by: Gluster Build System 
Reviewed-by: Niels de Vos 
Reviewed-by: Poornima G 
Reviewed-by: Prashanth Pai 
Reviewed-by: Raghavendra Talur

libgfapi: In glfs_resolve_at(), do not override the previous return value.

2014-02-20T01:09:30+00:00

Overriding ret to contain glfs_loc_touchup()s' return value implies that
if glfs_loc_touchup() is successful, glfs_resolve_at() is also successful
which is not necessarily true. This was causing glfs_resolve_at() to
succeed even if it couldn't resolve, thus create and other fops would
fail. Hence overriding ret only if glfs_loc_touchup() fails.

Change-Id: I0804afbd120b3798abe07e870bfc40bf162bc289
BUG: 1066837
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/7125
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

libgfapi: Don't ignore return value of glfs_loc_touchup()

2014-02-08T19:21:52+00:00

BUG: 789278
CID: 1124353

Change-Id: I7d2958cbc98faf45d723f17868c515762c50c618
Signed-off-by: Jose A. Rivera 
Reviewed-on: http://review.gluster.org/6931
Reviewed-by: Shyamsundar Ranganathan 
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

syncop: Change return value of syncop

2014-01-20T07:05:15+00:00

Problem:
We found a day-1 bug when syncop_xxx() infra is used inside a synctask with
compilation optimization (CFLAGS -O2).

Detailed explanation of the Root cause:
We found the bug in 'gf_defrag_migrate_data' in rebalance operation:

Lets look at interesting parts of the function:

int
gf_defrag_migrate_data (xlator_t *this, gf_defrag_info_t *defrag, loc_t *loc,
                        dict_t *migrate_data)
{
.....
code section - [ Loop ]
        while ((ret = syncop_readdirp (this, fd, 131072, offset, NULL,
                                       &entries)) != 0) {
.....
code section - [ ERRNO-1 ] (errno of readdirp is stored in readdir_operrno by a
thread)
                /* Need to keep track of ENOENT errno, that means, there is no
                   need to send more readdirp() */
                readdir_operrno = errno;
.....
code section - [ SYNCOP-1 ] (syncop_getxattr is called by a thread)
                        ret = syncop_getxattr (this, &entry_loc, &dict,
                                               GF_XATTR_LINKINFO_KEY);
code section - [ ERRNO-2]   (checking for failures of syncop_getxattr(). This
may not always be executed in same thread which executed [SYNCOP-1])
                        if (ret < 0) {
                                if (errno != ENODATA) {
                                        loglevel = GF_LOG_ERROR;
                                        defrag->total_failures += 1;
.....
}

the function above could be executed by thread(t1) till [SYNCOP-1] and code
from [ERRNO-2] can be executed by a different thread(t2) because of the way
syncop-infra schedules the tasks.

when the code is compiled with -O2 optimization this is the assembly code that
is generated:
 [ERRNO-1]
1165                        readdir_operrno = errno; <<---- errno gets expanded
as *(__errno_location())
   0x00007fd149d48b60 <+496>:        callq  0x7fd149d410c0 
   0x00007fd149d48b72 <+514>:        mov    %rax,0x50(%rsp) <<------ Address
returned by __errno_location() is stored in a special location in stack for
later use.
   0x00007fd149d48b77 <+519>:        mov    (%rax),%eax
   0x00007fd149d48b79 <+521>:        mov    %eax,0x78(%rsp)
....
 [ERRNO-2]
1281                                        if (errno != ENODATA) {
   0x00007fd149d492ae <+2366>:        mov    0x50(%rsp),%rax <<-----  Because
it already stored the address returned by __errno_location(), it just
dereferences the address to get the errno value. BUT THIS CODE NEED NOT BE
EXECUTED BY SAME THREAD!!!
   0x00007fd149d492b3 <+2371>:        mov    $0x9,%ebp
   0x00007fd149d492b8 <+2376>:        mov    (%rax),%edi
   0x00007fd149d492ba <+2378>:        cmp    $0x3d,%edi

The problem is that __errno_location() value of t1 and t2 are different. So
[ERRNO-2] ends up reading errno of t1 instead of errno of t2 even though t2 is
executing [ERRNO-2] code section.

When code is compiled without any optimization for [ERRNO-2]:
1281                                        if (errno != ENODATA) {
   0x00007fd58e7a326f <+2237>:        callq  0x7fd58e797300
<<--- As it is calling __errno_location() again it gets the
location from t2 so it works as intended.
   0x00007fd58e7a3274 <+2242>:        mov    (%rax),%eax
   0x00007fd58e7a3276 <+2244>:        cmp    $0x3d,%eax
   0x00007fd58e7a3279 <+2247>:        je     0x7fd58e7a32a1


Fix:
Make syncop_xxx() return (-errno) value as the return value in
case of errors and all the functions which make syncop_xxx() will need to use
(-ret) to figure out the reason for failure in case of syncop_xxx() failures.

Change-Id: I314d20dabe55d3e62ff66f3b4adb1cac2eaebb57
BUG: 1040356
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/6475
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

gfapi: remove unnecessary call to glfs_resolve_base()

2013-11-08T07:55:59+00:00

Calling glfs_resolve_base() on the root inode for every resolver
invocation is unnecessary and wasteful.

Here are the results from running a test program which performs
path based operations (creates and deletes 1000 files):

Without patch:
[root@blackbox ~]# sync
[root@blackbox ~]# time ./a.out 1

real    0m4.314s
user    0m1.923s
sys     0m1.144s
[root@blackbox ~]# sync
[root@blackbox ~]# time ./a.out 1

real    0m4.383s
user    0m1.940s
sys     0m1.177s
[root@blackbox ~]# sync
[root@blackbox ~]# time ./a.out 1

real    0m4.339s
user    0m1.863s
sys     0m1.129s

With patch:

[root@blackbox ~]# sync
[root@blackbox ~]# time ./a.out 1

real    0m3.005s
user    0m1.162s
sys     0m0.816s
[root@blackbox ~]# sync
[root@blackbox ~]# time ./a.out 1

real    0m3.188s
user    0m1.222s
sys     0m0.867s
[root@blackbox ~]# sync
[root@blackbox ~]# time ./a.out 1

real    0m2.999s
user    0m1.131s
sys     0m0.832s

Change-Id: Id160a24f44b4dccfcfce99a6f69ddb8938523cd5
BUG: 953694
Signed-off-by: Anand Avati 
Reviewed-on: http://review.gluster.org/6131
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra Talur 
Reviewed-by: Raghavendra Bhat 
Reviewed-by: Vijay Bellur

gfapi: wake migration waiters after migration

2013-10-22T16:33:39+00:00

Wake call threads which are waiting on migration to complete with
a cond_broadcast. Else if any call which arrives right when migration
is attempted will end up hanging indefinitely.

Change-Id: I7df5298f93998d9a54fb12c16654e62333018ece
BUG: 953694
Signed-off-by: Anand Avati 
Reviewed-on: http://review.gluster.org/6111
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra Bhat 
Reviewed-by: Shyamsundar Ranganathan

gfapi: object handle based API extensions

2013-10-11T19:16:46+00:00

There is an ongoing effort to integrate NFS Ganesha (
https://github.com/nfs-ganesha/nfs-ganesha/wiki ) with GlusterFS as one of
the file system back ends.

Towards this we need extensions to gfapi that can handle object based
operations. Meaning, instead of using full paths or relative paths from
cwd, it is required that we can work with APIs, like the *at POSIX
variants, to be able to create, lookup, open etc. files and directories.
Hence the objects are the files or directories themselves and we give out
handles to these objects that can be used for further operations.

This code drop is an initial implementation of the proposed APIs.

The new APIs are implemented as glfs_h_XXX variants in the file
glfs-handleops.c to mirror glfs-fops.c style. The code leverages holding
onto inode references and doling these out as opaque/cookie type objects to
the callers, to enable them to be used as handles in other operations.

An fd based approach was considered, but due to the extra footprint that
the fd structure and its counterparts would incur, this was dropped to take
the approach of holding inode references themselves.

Tested by extending glfsxmp.c to invoke and exercise the added APIs, and
further tested with a reference integration of the same as an FSAL with NFS
Ganesha.

Change-Id: I23629c99e905b54070fa2e6565147812e5f3fa5d
BUG: 1016000
Signed-off-by: R.Shyamsundar 
Reviewed-on: http://review.gluster.org/5936
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

gfapi: store the open/create flags in fd_t object

2013-09-06T22:56:57+00:00

The flags passed on to open and create calls were not being saved
in the fd_t object, hence the fd migration was failing.

Change-Id: I486bb818477fe4c393d64a711534a082162a0e53
BUG: 1005159
Signed-off-by: Poornima 
Reviewed-on: http://review.gluster.org/5837
Reviewed-by: Amar Tumballi 
Reviewed-by: Raghavendra Bhat 
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

gfapi: apply an upper bound on nested symlink resolution

2013-09-05T12:26:30+00:00

In case of nested symlink resolution, implement an upper
bound on the number of such nested levels the resolver will
descend. This limit is arbitrary, and set to 2048 nested
levels.

Change-Id: I264e5bd60d317eda97f4e6f49bd5d8694f8664a9
BUG: 1004100
Signed-off-by: Anand Avati 
Reviewed-on: http://review.gluster.org/5768
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra Talur 
Reviewed-by: Vijay Bellur