Gluster users - Sep 2014 - Feedback and future plans for glusterfsiostat

Hi all,

As you might know, I completed my GSOC '14 project : glusterfsiostat under
the mentorship of Krishnan Parathasarthi. I'm glad for the support and
guidance from the Gluster community.

The next task is to plan further development of the tool and make it even
better. So, just looking for some feedback and suggestions. The work done till
now can majorly be divided into 3 parts:

1) io-stats patch to display FOP information through meta : 
http://review.gluster.org/#/c/8244/
Please do review this so that it can be merged.

2) stat.py script which is a command line utility similar to nfsiostat which
shows the basic read/write speed among other ops related values.

3) server.py which is a web server written with very basic python libraries.
I've attached some screenshots of the server in action while visualizing
live data from gluster mounts in the form of web based graphs.

Justin, since this web server implementation is a bit similar to your tool
glusterflow, your feedback regarding this is really important and valuable.

Some key points for those who want to test this thing.

	* Apply this patch http://review.gluster.org/#/c/8244/ , build and install.
	* Please grab the latest source code from
https://forge.gluster.org/glusterfsiostat
	* Profiling needs to be turned on for the volumes regarding which you want I/O
data.
	* Run the cmd line tool by `python stat.py` Giving --help tag would list the
other options.
	* Start the server.py script in the same way and point your browser at
'localhost:8080'
	* The gluster mounts would be visible as separate tabs on the left in browser.
Click on a tab to see the related graphs.
>	* Since these graphs show live I/O activity, you need to run a read/write
operation to see the graphs in action.
Please do contact me regarding any suggestions or thoughts about this.

Regards
Vipul Nayyar 
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140903/d4c087b1/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screenshot from 2014-08-18 20:38:39.png
Type: image/png
Size: 108879 bytes
Desc: not available
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140903/d4c087b1/attachment-0004.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screenshot from 2014-08-18 20:38:52.png
Type: image/png
Size: 96851 bytes
Desc: not available
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140903/d4c087b1/attachment-0005.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screenshot from 2014-08-18 20:38:58.png
Type: image/png
Size: 107599 bytes
Desc: not available
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140903/d4c087b1/attachment-0006.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screenshot from 2014-08-18 20:39:03.png
Type: image/png
Size: 94213 bytes
Desc: not available
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140903/d4c087b1/attachment-0007.png>

On 03/09/2014, at 3:48 PM, Vipul Nayyar wrote:
<snip>
> 3) server.py which is a web server written with very basic python
libraries. I've attached some screenshots of the server in action while
visualizing live data from gluster mounts in the form of web based graphs.
> 
> Justin, since this web server implementation is a bit similar to your tool
glusterflow, your feedback regarding this is really important and valuable.
Thanks.  Server.py is the bit that caught my attention too. :)

I like the screenshots.  Does server.py update/refresh in real-time?

Also, one thing that's missing in the screenshots is the time info
on the X axis.  eg how far apart are each of the tick marks on the
x axis?  Minor oversight, but important. ;)

> Some key points for those who want to test this thing.
> 
> ? Apply this patch http://review.gluster.org/#/c/8244/ , build and install.
> ? Please grab the latest source code from
https://forge.gluster.org/glusterfsiostat
> ? Profiling needs to be turned on for the volumes regarding which you want
I/O data.
> ? Run the cmd line tool by `python stat.py` Giving --help tag would list
the other options.
> ? Start the server.py script in the same way and point your browser at
'localhost:8080'
> ? The gluster mounts would be visible as separate tabs on the left in
browser. Click on a tab to see the related graphs.
> ? Since these graphs show live I/O activity, you need to run a read/write
operation to see the graphs in action.
> Please do contact me regarding any suggestions or thoughts about this.
This is is very cool.  As a thought, since I don't know the code
at all, is could it do stuff for parts of a volume?

For example in the server.py GUI a person could give a directory
path inside a volume, and it would show the IO operations stats
for just that path?

+ Justin

--
GlusterFS - http://www.gluster.org

An open source, distributed file system scaling to several
petabytes, and handling thousands of clients.

My personal twitter: twitter.com/realjustinclift