search for: hdfs

...sues I can't figure out why they happen. I use RHEL5.7, packetlist: rpm -qa|grep samba samba-3.0.33-3.29.el5_7.4.x86_64 samba-common-3.0.33-3.29.el5_7.4.x86_64 2 servers, one provides samba-shares and one I use as client. Samba-Server (HOST2): FuseFS: hadoop-fuse-dfs#dfs://HOSTNAME:9000 /123/hdfs fuse usetrash,rw 0 0 mount: fuse on /123/hdfs type fuse (rw,nosuid,nodev,allow_other,default_permissions) the mount /123 is accessible over ssh, cp, rsnyc from any host. Now I was using NFS for populating, but the performance was not as I expected. So I switched to samba and get real good results...

hi all, i am new to r , i am trying to run data mining algorithms using map reduce framework.. * *i have few basic doubts* *1. can i give file in hdfs to kmeans( ) ? ?I tried as > file1 = hdfs.file("testdata/synthetic_control.data") > isf = hdfs.read(lsf,5242880,0) >l = kmeans(isf,2,10) its not working.. please help 2. How to access the file in hdfs and give as input to algorithms ? please reply.. Regards, ./T.karthik* *...

Hello, I am trying to write the default "OrchardSprays" R data frame into HDFS using the "rhdfs" package. I want to write this data frame directly into HDFS without first storing it into any file in local file system. Which rhdfs command i should use? Can some one help me? I am very new to R and rhdfs. Regards, Gaurav [[alternative HTML version deleted]]

Hello, I am trying to write the default "OrchardSprays" R data frame into HDFS using the "rhdfs" package. I want to write this data frame directly into HDFS without first storing it into any file in local file system. Which rhdfs command i should use? Can some one help me? I am very new to R and rhdfs. Regards, Gaurav [[alternative HTML version deleted]]

Hi, I have requirement to move the data from Linux local path( ex /home/user/sample.txt) to hadoop HDFS using RODBC in R I knew that we can move the data using rhive comamnds like *rhive.put* and *rhive.get *but looking for similar commands using RODBC as well. I would appreciate for your inputs. Regards, Divakar Phoenix, USA [[alternative HTML version deleted]]

...lient. All of this is 100% > open source. > >> 4. metadata machines should be redundant (or at least backup medatada host >> should be manually convertible into master metadata host if fatal failure >> to master or corruption of its data happens) >> Sounds like Hadoop HDFS might be worth looking into: https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsUserGuide.html#Overview >> >> What I would like to avoid/exclude: >> >> 1. Proprietary commercial solutions, as: >> >> a. I would like to stay on as minimal...

Hi, all, i try to check the performance of Map/Reduce of Gluster File system. Mapper side speed is quite good and it is sometimes faster than hadoop's map job. But in the Reduce Side job is much slower than hadoop. i analyze the result and i found the primary reason of slow speed is bad performance in Merging stage. Would you have any suggestion for this issue FYI check the blog

Hi, I have MP3 files stored inside hadoop's HDFS. Is there anyway of running icecast on top of those MP3 files ? Is there any help on how to do it? Thanks, jb -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.xiph.org/pipermail/icecast/attachments/20091109/9dc520bb/attachment.htm

Hi: Recently I have some study on linux container(lxc), which is a light weight OS level virtualization. With the previous knowledge of tapdisk, I have an assumption that, I may could use the vhd for each container to seperate the storage of all containers, or even in later days, vhd is stored in distributed storage, container could be migrated. Build filesystem on tapdev

Do we have enormous fragmentation here on our X4500 with Solaris 10, ZFS Version 10? What except zfs send/receive can be done to free the fragmented space? One ZFS was used for some month to store some large disk images (each 50GByte large) which are copied there with rsync. This ZFS then reports 6.39TByte usage with zfs list and only 2TByte usage with du. The other ZFS was used for similar

Hi all, I am just trying to consider my options for storing a large mass of data (tens of terrabytes of files) and one idea is to build a clustered FS of some kind. Has anybody had any experience with that? Any recommendations? Thanks in advance for any and all advice. Boris.

Hi all, Has anyone created a Xen cluster to run a hadoop vm cluster? I would be interested in how it performs Thanks Lance -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users

...c(allfiles,"/home/sguha/tmp/x/build/jar/rhipe.jar") .jinit(classpath=allfiles) Now, I create a new object .jnew("org/guha/SeqKeyList",.jnull) but get this error Exception in thread "main" java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.hadoop.hdfs.DistributedFileSystem at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:720) This does not appear to be possible(ClassNotFound), since hadoop-0.19-core.jar which is *present* in the output of .jclassPath() does contain an entry to this class. Moreover, when run at the command li...

All, I am using a pretty crude method to get data out of HDFS via Hive and into R and was curious about alternatives that the group has explored. Basically, I run a system command that runs a hive statement and writes the returned data to a delimited file. Then, I read that file into an object and continue. For example: hive.script <- "select * from...

...ented in the PHP client.Warning: type 7 is currently not implemented in the PHP client.Warning: type 7 is currently not implemented in the PHP client.Warning: type 7 is currently not implemented in the PHP client.Warning: type 7 is currently not implemented in the PHP client. Array localhost Array hdfs://localhost:54310 ----- Sagar Nikam B.Pharm, M.Sc(Bioinformatics) Software Engineer (Data Research Analyst ) Trendwise Analytics,Bangalore India -- View this message in context: http://r.789695.n4.nabble.com/Rserve-PHP-client-warning-tp4647110.html Sent from the R help mailing list archive a...

I generally run TestDFSIO for Hadoop running on HDFS as shown: hadoop jar hadoop-*test*.jar TestDFSIO -write -nrFiles 10 -fileSize 1000 How to run it on lustre filesystem? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20130509/821dc4bb/attachment.html

...; is a new on-line interactive tutorial for R: http://bit.ly/W0Aqr2 RStudio's "shiny" package for interactive web-based R applications is now available on CRAN: http://bit.ly/W0Aqr3 Revolution R Enterprise 6.1 adds the capability for fitting rpart-style tree models to data in Hadoop HDFS: http://bit.ly/W0Aqr4 A New York Times graphic editor shares how he uses R base graphics to create bar charts in the newspaper: http://bit.ly/W0Aszk Some non-R stories in the past month included: how to make a 360-degree photo of a landmark (http://bit.ly/W0Aszl), influential data scientists on T...

...esides:* * .jinit("pkgPath")* * * * # Adds all the required JAR to the class path:* * .jaddClassPath(paste(pkgPath, "Eda.jar", sep="/"))* * .jaddClassPath(paste(pkgPath, "commons-cli-1.2.jar", sep="/"))* * .jaddClassPath(paste(pkgPath, "hadoop-hdfs-2.0.0-cdh4.3.0.jar", sep="/"))* * .jaddClassPath(paste(pkgPath, "slf4j-log4j12-1.6.1.jar", sep="/"))* * .jaddClassPath(paste(pkgPath, "commons-configuration-1.6.jar", sep="/"))* * .jaddClassPath(paste(pkgPath, "guava-11.0.2.jar", sep=...

...age to make interactive web-based applications with R: http://bit.ly/TMFMVd John Deere uses R to speed up the production of tractors and to forecast crop yields, as shown in this webinar replay: http://bit.ly/TMFP3g Revolution R Enterprise 6.1 is now available, with big-data tree models, a Hadoop HDFS connection, and improved performance: http://bit.ly/TMFP3j Another take on of using ggplot2 to visualize the DW-NOMINATE data on ideology shifts in the US Congress: http://bit.ly/TMFMVf I present the Data Scientist's Tookit (including R, of course) in this replay of my webinar, The Rise of Da...

Dear Experts, I would like to ask everybody: what would you advise to use as a storage cluster, or as a distributed filesystem. I made my own research of what I can do, but I hit a snag with my seemingly best choice, so I decided to stay away from it finally, and ask clever people what they would use. My requirements are: 1. I would like to have one big (say, comparable to petabyte)