glusterfs.git/cli/src/cli-cmd-volume.c, branch v3.5.6

features/quota : Fix XML output for quota list command

2015-07-07T16:11:38+00:00

This is a backport of http://review.gluster.org/#/c/9481/

> Sample output:
> ---------------
>
> Sample 1)
> ----------
> [root@snapshot-28 glusterfs]# gluster volume quota vol1 list /dir1 /dir4
> /dir5 --xml
> 
> 
>   0
>   0
>   
>   
>     
>       /dir1
>       10.0MB
>       80%
>       0Bytes
>       10.0MB
>     
>     
>       /dir4
>       No such file or directory
>     
>     
>       /dir5
>       No such file or directory
>     
>   
> 
>
> Sample 2)
> ---------
> gluster volume quota vol1 list --xml
> 
> 
>   0
>   0
>   
>   
> 
> 
> 
>   
>     
>       /dir
>       10.0MB
>       80%
>       0Bytes
>       10.0MB
>     
>     
>       /dir1
>       10.0MB
>       80%
>       0Bytes
>       10.0MB
>     
>   
> 
>
> Change-Id: I8a8d83cff88f778e5ee01fbca07d9f94c412317a
> BUG: 1185259
> Signed-off-by: Sachin Pandit 
> Reviewed-on: http://review.gluster.org/9481
> Reviewed-by: Vijaikumar Mallikarjuna 
> Tested-by: Gluster Build System 
> Reviewed-by: Kaushal M 

Change-Id: Ibdf51db626a07e68b5ace98140877f6d21918c20
BUG: 1231641
Signed-off-by: vmallika 
Reviewed-on: http://review.gluster.org/11220
Tested-by: Gluster Build System 
Reviewed-by: Sachin Pandit 
Reviewed-by: Niels de Vos

NetBSD build fixes for release-3.5

2014-06-08T09:18:13+00:00

1) Make sure __THROW is definedThis is a backport of I6e7cb1eb59b84988e155e9a8b696e842b7ff8f7f

2) include  before  so that XDR is defined
This was fixed in master within I20193d3f8904388e47344e523b3787dbeab044acbut weonly pull up

3) NetBSD's gettext is in libintl, hence search it at configure time
This is a backport of I651a74fe49c3f087fe135dab3453fd5b18b4268a

4) include  to have WEXITSTATUS defined
This problem does not exist in master as WEXITSTATUS is not used

5) Do not define popcountl() on NetBSD as it is in 
This is a backport of I4428a88b1e0d7c5f6740022861ffe230dbbd84bd

BUG: 764655
Change-Id: Ieea5a2a627e2b7930525d6c525f1602073574a97
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/7925
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Niels de Vos 
Tested-by: Gluster Build System

heal: glfs-heal implementation

2014-04-28T16:39:39+00:00

Change-Id: I634f24d90cd856ceab3cc0c6e9a91003f443403e
BUG: 1039544
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/6511
Tested-by: Gluster Build System 
Reviewed-by: Ravishankar N 
Reviewed-by: Vijay Bellur

cli, glusterd: More quota fixes ...

2013-12-03T09:48:31+00:00

... which may be grouped under the following categories:

1. Fix incorrect cli exit status for 'quota list' cmd
2. Print appropriate error message on quota parse errors in cli

        Authored by: Anuradha Talur 

3. glusterd: Improve quota validation during stage-op
4. Fix peer probe issues resulting from quota conf checksum mismatches
5. Enhancements to CLI output in the event of quota command failures

        Authored by: Kaushal Madappa 

7. Move aux mount location from /tmp to /var/run/gluster

        Authored by: Krishnan Parthasarathi 

8. Fix performance issues in quota limit-usage

        Authored by: Krutika Dhananjay 

Note: Some functions that were used in earlier version of quota,
      that aren't called anymore have been removed.

Change-Id: I963d4145f3ecdfe30c61bfa8920baccb33d2d4bd
BUG: 969461
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/6386
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

cli/glusterd: Changes to quota command Quota feature

2013-11-26T18:25:27+00:00

 re-work.

Following are the cli commands that are new/re-worked:
======================================================

volume quota  {enable|disable|list [ ...]|remove | default-soft-limit } |
volume quota  {limit-usage   []} |
volume quota  {alert-time|soft-timeout|hard-timeout} {}
volume status [all |  [nfs|shd||quotad]] [detail|clients|mem|inode|fd|callpool]
volume statedump  [nfs|quotad] [all|mem|iobuf|callpool|priv|fd|inode|history]

glusterd changes:
=================
* Quota limits are now set as extended attributes by glusterd from
  the aux mount created by the cli.
* The gfids of the directories on which quota limits are set
  for a given volume are stored in
  /var/lib/glusterd/vols//quota.conf file in binary format,
  and whose cksum and version is stored in
  /var/lib/glusterd/vols//quota.cksum.

Original-author: Krutika Dhananjay 
Original-author: Krishnan Parthasarathi 

BUG: 969461
Change-Id: If32bba36c67f9c2a30417af9c6389045b2b7c13b
Signed-off-by: Krutika Dhananjay 
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/6003
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

bd: posix/multi-brick support to BD xlator

2013-11-13T19:38:42+00:00

Current BD xlator (block backend) has a few limitations such as
* Creation of directories not supported
* Supports only single brick
* Does not use extended attributes (and client gfid) like posix xlator
* Creation of special files (symbolic links, device nodes etc) not
  supported

Basic limitation of not allowing directory creation is blocking
oVirt/VDSM to consume BD xlator as part of Gluster domain since VDSM
creates multi-level directories when GlusterFS is used as storage
backend for storing VM images.

To overcome these limitations a new BD xlator with following
improvements is suggested.

* New hybrid BD xlator that handles both regular files and block device
  files
* The volume will have both POSIX and BD bricks. Regular files are
  created on POSIX bricks, block devices are created on the BD brick (VG)
* BD xlator leverages exiting POSIX xlator for most POSIX calls and
  hence sits above the POSIX xlator
* Block device file is differentiated from regular file by an extended
  attribute
* The xattr 'user.glusterfs.bd' (BD_XATTR) plays a role in mapping a
  posix file to Logical Volume (LV).
* When a client sends a request to set BD_XATTR on a posix file, a new
  LV is created and mapped to posix file. So every block device will
  have a representative file in POSIX brick with 'user.glusterfs.bd'
  (BD_XATTR) set.
* Here after all operations on this file results in LV related
  operations.

For example opening a file that has BD_XATTR set results in opening
the LV block device, reading results in reading the corresponding LV
block device.

When BD xlator gets request to set BD_XATTR via setxattr call, it
creates a LV and information about this LV is placed in the xattr of the
posix file. xattr "user.glusterfs.bd" used to identify that posix file
is mapped to BD.

Usage:
Server side:
[root@host1 ~]# gluster volume create bdvol host1:/storage/vg1_info?vg1 host2:/storage/vg2_info?vg2
It creates a distributed gluster volume 'bdvol' with Volume Group vg1
using posix brick /storage/vg1_info in host1 and Volume Group vg2 using
/storage/vg2_info in host2.

[root@host1 ~]# gluster volume start bdvol

Client side:
[root@node ~]# mount -t glusterfs host1:/bdvol /media
[root@node ~]# touch /media/posix
It creates regular posix file 'posix' in either host1:/vg1 or host2:/vg2 brick
[root@node ~]# mkdir /media/image
[root@node ~]# touch /media/image/lv1
It also creates regular posix file 'lv1' in either host1:/vg1 or
host2:/vg2 brick
[root@node ~]# setfattr -n "user.glusterfs.bd" -v "lv" /media/image/lv1
[root@node ~]#
Above setxattr results in creating a new LV in corresponding brick's VG
and it sets 'user.glusterfs.bd' with value 'lv: --deltag
>

Changes from previous version V5:
* Removed support for delayed deleting of LVs

Changes from previous version V4:
* Consolidated the patches
* Removed usage of BD_XATTR_SIZE and consolidated it in BD_XATTR.

Changes from previous version V3:
* Added support in FUSE to support full/linked clone
* Added support to merge snapshots and provide information about origin
* bd_map xlator removed
* iatt structure used in inode_ctx. iatt is cached and updated during
fsync/flush
* aio support
* Type and capabilities of volume are exported through getxattr

Changes from version 2:
* Used inode_context for caching BD size and to check if loc/fd is BD or
  not.
* Added GlusterFS server offloaded copy and snapshot through setfattr
  FOP. As part of this libgfapi is modified.
* BD xlator supports stripe
* During unlinking if a LV file is already opened, its added to delete
  list and bd_del_thread tries to delete from this list when a last
  reference to that file is closed.

Changes from previous version:
* gfid is used as name of LV
* ? is used to specify VG name for creating BD volume in volume
  create, add-brick. gluster volume create volname host:/path?vg
* open-behind issue is fixed
* A replicate brick can be added dynamically and LVs from source brick
  are replicated to destination brick
* A distribute brick can be added dynamically and rebalance operation
  distributes existing LVs/files to the new brick
* Thin provisioning support added.
* bd_map xlator support retained
* setfattr -n user.glusterfs.bd -v "lv" creates a regular LV and
  setfattr -n user.glusterfs.bd -v "thin" creates thin LV
* Capability and backend information added to gluster volume info (and
--xml) so
  that management tools can exploit BD xlator.
* tracing support for bd xlator added

TODO:
* Add support to display snapshots for a given LV
* Display posix filename for list-origin instead of gfid

Change-Id: I00d32dfbab3b7c806e0841515c86c3aa519332f2
BUG: 1028672
Signed-off-by: M. Mohan Kumar 
Reviewed-on: http://review.gluster.org/4809
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

bd_map: Remove bd_map xlator

2013-11-13T19:38:28+00:00

Remove bd_map xlator and CLI related changes.

Change-Id: If7086205df1907127c1a1fa4ba603f1c48421d09
BUG: 1028672
Signed-off-by: M. Mohan Kumar 
Reviewed-on: http://review.gluster.org/5747
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

glusterd: modify remove-brick CLI message.

2013-11-08T07:54:26+00:00

In the current context "replica_cnt" is used just to know whether the
specific key exists or not by calling "dict_get_int32", which we can
replace by "dict_get ()". And changing the log message as it is more
appropriate to say "migration of data" rather than "rebalance".

This patch refactors commit 51c6fa7a354826744de98 against BZ 961669

reviewed on : http://review.gluster.org/5566

Change-Id: I48eae206a28d4083975e64407ed8fe4539f9c24b
BUG: 1027270
Signed-off-by: Ravishankar N 
Original patch: Susant Palai 
Reviewed-on: http://review.gluster.org/6001
Tested-by: Gluster Build System 
Reviewed-by: Krishnan Parthasarathi 
Reviewed-by: susant palai 
Reviewed-by: Vijay Bellur

cluster/afr: [Feature] Command implementation to get heal-count

2013-10-14T21:41:54+00:00

Currently to know the number of files to be healed, either user
has to go to backend and check the number of entries present in
indices/xattrop directory. But if a volume consists of large
number of bricks, going to each backend and counting the number
of entries is a time-taking task. Otherwise user can give
gluster volume heal vol-name info command but with this
approach if no. of entries are very hugh in the indices/
xattrop directory, it will comsume time.

So as a feature, new command is implemented.

Command 1: gluster volume heal vn statistics heal-count
This command will get the number of entries present in
every brick of a volume. The output displays only entries
count.

Command 2: gluster volume heal vn statistics heal-count
           replica 192.168.122.1:/home/user/brickname

           Here if we are concerned with just one replica.
So providing any one of the brick of a replica will get
the number of entries to be healed for that replica only.

Example:
Replicate volume with replica count 2.

Backend status:
--------------
[root@dhcp-0-17 xattrop]# ls -lia | wc -l
1918

NOTE: Out of 1918, 2 entries are  dummy
entries so actual no. of entries to be healed are
1916.

[root@dhcp-0-17 xattrop]# pwd
/home/user/2ty/.glusterfs/indices/xattrop

Command output:
--------------
Gathering count of entries to be healed on volume volume3 has been successful

Brick 192.168.122.1:/home/user/22iu
Status: Brick is Not connected
Entries count is not available

Brick 192.168.122.1:/home/user/2ty
Number of entries: 1916

Change-Id: I72452f3de50502dc898076ec74d434d9e77fd290
BUG: 1015990
Signed-off-by: Venkatesh Somyajulu 
Reviewed-on: http://review.gluster.org/6044
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

cluster/afr : Implementation of command "gluster volume heal vn statistics"

2013-10-14T21:41:44+00:00

"gluster volume heal volumename statistics" command gives the summary
of the afr crawl done based on the entries present in the xattrop
directory. Whenever afr crawls are attempted, the beginning time of
crawl, end time of crawl, no of files healed, heal-failed count and
number of files in split brain are shown along with the type of the
crawl. If crawl is already in progress then it will give the number
of files healed, heal failed count and number of files in split-brain
from the beginning of the crawl and instead of telling the end time of
the crawl, "CRAWL IN PROGRESS" message will be shown.

Output format:
command: "gluster volume heal volume-name statistics"
Output:
Gathering afr crawl statistics crawl statistics on volume volume-name
has been successful
------------------------------------------------

Crawl statistics for brick no 0
Hostname of brick 192.168.122.248

Starting time of crawl: Wed Jul 10 15:52:38 2013

Ending time of crawl: Wed Jul 10 15:52:38 2013

Type of crawl: INDEX
No. of entries healed: 0
No. of entries in split-brain: 0
No. of heal failed entries: 0

Starting time of crawl: Wed Jul 10 15:52:38 2013

Ending time of crawl: Wed Jul 10 15:52:38 2013

Type of crawl: INDEX
No. of entries healed: 0
No. of entries in split-brain: 0
No. of heal failed entries: 0

------------------------------------------------

Crawl statistics for brick no 1
Hostname of brick 192.168.122.1

Starting time of crawl: Wed Jul 10 15:52:42 2013

Ending time of crawl: Wed Jul 10 15:52:42 2013

Type of crawl: INDEX
No. of entries healed: 0
No. of entries in split-brain: 0
No. of heal failed entries: 0

Starting time of crawl: Wed Jul 10 15:52:42 2013

Ending time of crawl: Wed Jul 10 15:52:42 2013

Type of crawl: INDEX
No. of entries healed: 0
No. of entries in split-brain: 0
No. of heal failed entries: 0

--------------------------------------------------

Change-Id: I10bf9d10b005741db9973fb1352e0dd59ed99aa9
BUG: 949400
Signed-off-by: Venkatesh Somyajulu 
Reviewed-on: http://review.gluster.org/4790
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati