thr3ads.net - R help - [R] remove a "corrupted file" after using download.file() with R on Windows 7 [Sep 2016]

If this information is useful, please help other people find it:
Share via:

Fabien Tarrade

2016-Sep-29 04:32 UTC

[R] remove a "corrupted file" after using download.file() with R on Windows 7

Hi there,

Sometime download.file() failed to download the file and I would like to 
remove the correspond file.
The issue is that I am not able to do it and Windows complain that the 
file is use by another application.
I try to closeAllConnections(), or unlink() before removing the file but 
without sucess.

Any idea how I should proceed &

Please find the code below

  # consider warning as an error
   options(warn=2)

   # try to download the file
   tryCatch({
     download.file(url,path_file,mode="wb",quiet=quiet)
     return(0)
   },error = function(e){
     if(verbose){
       print(e)
       print(e$message)
     }
     # close file when it failed
     if (file.exists(path_file)){
       closeAllConnections()
       #unlink(path_file, recursive=TRUE)
       #file.create(path_file,overwrite=TRUE,showWarning=TRUE)
       #system(paste0('open "', path_file, '"'))
       file.remove(path_file,overwrite=TRUE,showWarning=TRUE)
     }
     return(1)
     }
)

Thanks a lot
Cheers
Fabien

-- 
Dr Fabien Tarrade

Quantitative Analyst/Developer - Data Scientist

Senior data analyst specialised in the modelling, processing and 
statistical treatment of data.
PhD in Physics, 10 years of experience as researcher at the forefront of 
international scientific research.
Fascinated by finance and data modelling.

Geneva, Switzerland

Email : contact at fabien-tarrade.eu <mailto:contact at fabien-tarrade.eu>
Phone : www.fabien-tarrade.eu <http://www.fabien-tarrade.eu>
Phone : +33 (0)6 14 78 70 90

LinkedIn <http://ch.linkedin.com/in/fabientarrade/> Twitter 
<https://twitter.com/fabtar> Google 
<https://plus.google.com/+FabienTarradeProfile/posts> Facebook 
<https://www.facebook.com/fabien.tarrade.eu> Google 
<skype:fabtarhiggs?call> Xing
<https://www.xing.com/profile/Fabien_Tarrade>

Robert Baer

2016-Sep-29 19:35 UTC

head link

[R] remove a "corrupted file" after using download.file() with R on Windows 7

On 9/28/2016 11:32 PM, Fabien Tarrade wrote:> Hi there,
>
> Sometime download.file() failed to download the file and I would like 
> to remove the correspond file.No answers, but a couple of additional questions:
1)  Does the issue persist if you close R or does the file remain locked 
against deletion?
2) If so, is there a related process in the task list if you use 
CTRL-ALT-DEL?
3) Does       print(e$message) yield any useful information when it hangs?

Would debugging in R Studio shed additional light?
> The issue is that I am not able to do it and Windows complain that the 
> file is use by another application.
> I try to closeAllConnections(), or unlink() before removing the file 
> but without sucess.
>
> Any idea how I should proceed &
>
> Please find the code below
>
>  # consider warning as an error
>   options(warn=2)
>
>   # try to download the file
>   tryCatch({
>     download.file(url,path_file,mode="wb",quiet=quiet)
>     return(0)
>   },error = function(e){
>     if(verbose){
>       print(e)
>       print(e$message)
>     }
>     # close file when it failed
>     if (file.exists(path_file)){
>       closeAllConnections()
>       #unlink(path_file, recursive=TRUE)
>       #file.create(path_file,overwrite=TRUE,showWarning=TRUE)
>       #system(paste0('open "', path_file, '"'))
>       file.remove(path_file,overwrite=TRUE,showWarning=TRUE)
>     }
>     return(1)
>     }
> )
>
> Thanks a lot
> Cheers
> Fabien
>
-- 


--
Robert W. Baer, Ph.D.
Professor of Physiology
Kirksville College of Osteopathic Medicine
A T Still University of Health Sciences
800 W. Jefferson St
Kirksville, MO 63501
660-626-2321 Department
660-626-2965 FAX

Henrik Bengtsson

2016-Sep-30 02:22 UTC

head link

[R] remove a "corrupted file" after using download.file() with R on Windows 7

1. It could be that a virus checker locks the file.

2. There are Windows software tools that identify which process locks
a particular file, e.g. LockHunter (http://lockhunter.com/).  Those
should help you figure out what's going on.

3. R.utils::downloadFile() tries it's best to download files
atomically, i.e. it either gives you a fully downloaded file or not
all.  In your case, you might still end up with a temporary corrupt
file, but at least it will have a filename that is different than the
one you ask for.

Hope this helps

/Henrik

On Wed, Sep 28, 2016 at 9:32 PM, Fabien Tarrade
<fabien.tarrade at gmail.com> wrote:> Hi there,
>
> Sometime download.file() failed to download the file and I would like to
> remove the correspond file.
> The issue is that I am not able to do it and Windows complain that the file
> is use by another application.
> I try to closeAllConnections(), or unlink() before removing the file but
> without sucess.
>
> Any idea how I should proceed &
>
> Please find the code below
>
>  # consider warning as an error
>   options(warn=2)
>
>   # try to download the file
>   tryCatch({
>     download.file(url,path_file,mode="wb",quiet=quiet)
>     return(0)
>   },error = function(e){
>     if(verbose){
>       print(e)
>       print(e$message)
>     }
>     # close file when it failed
>     if (file.exists(path_file)){
>       closeAllConnections()
>       #unlink(path_file, recursive=TRUE)
>       #file.create(path_file,overwrite=TRUE,showWarning=TRUE)
>       #system(paste0('open "', path_file, '"'))
>       file.remove(path_file,overwrite=TRUE,showWarning=TRUE)
>     }
>     return(1)
>     }
> )
>
> Thanks a lot
> Cheers
> Fabien
>
> --
> Dr Fabien Tarrade
>
> Quantitative Analyst/Developer - Data Scientist
>
> Senior data analyst specialised in the modelling, processing and
statistical
> treatment of data.
> PhD in Physics, 10 years of experience as researcher at the forefront of
> international scientific research.
> Fascinated by finance and data modelling.
>
> Geneva, Switzerland
>
> Email : contact at fabien-tarrade.eu <mailto:contact at
fabien-tarrade.eu>
> Phone : www.fabien-tarrade.eu <http://www.fabien-tarrade.eu>
> Phone : +33 (0)6 14 78 70 90
>
> LinkedIn <http://ch.linkedin.com/in/fabientarrade/> Twitter
> <https://twitter.com/fabtar> Google
> <https://plus.google.com/+FabienTarradeProfile/posts> Facebook
> <https://www.facebook.com/fabien.tarrade.eu> Google
<skype:fabtarhiggs?call>
> Xing <https://www.xing.com/profile/Fabien_Tarrade>
>
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

Fabien Tarrade

2016-Oct-02 16:48 UTC

head link

[R] remove a "corrupted file" after using download.file() with R on Windows 7

Hi Robert,

sorry for the delays>> Sometime download.file() failed to download the file and I would like 
>> to remove the correspond file.
> No answers, but a couple of additional questions:
> 1)  Does the issue persist if you close R or does the file remain 
> locked against deletion?
no, if I close R then I can remove the file> 2) If so, is there a related process in the task list if you use 
> CTRL-ALT-DEL?
nothing special a part from R studio> 3) Does       print(e$message) yield any useful information when it 
> hangs?I have an url to access the pages of .tiff documents. The issue that I 
don't know how many pages have each documents. So I try to copy and 
incrememt the index for the number of pges until it failed. I know it is 
not an idea solution but this is I have now until we get this info into 
a DB.> Would debugging in R Studio shed additional light?I will give a try.

Thanks
Cheers
Fabien

-- 
Dr Fabien Tarrade

Quantitative Analyst/Developer - Data Scientist

Senior data analyst specialised in the modelling, processing and 
statistical treatment of data.
PhD in Physics, 10 years of experience as researcher at the forefront of 
international scientific research.
Fascinated by finance and data modelling.

Geneva, Switzerland

Email : contact at fabien-tarrade.eu <mailto:contact at fabien-tarrade.eu>
Phone : www.fabien-tarrade.eu <http://www.fabien-tarrade.eu>
Phone : +33 (0)6 14 78 70 90

LinkedIn <http://ch.linkedin.com/in/fabientarrade/> Twitter 
<https://twitter.com/fabtar> Google 
<https://plus.google.com/+FabienTarradeProfile/posts> Facebook 
<https://www.facebook.com/fabien.tarrade.eu> Google 
<skype:fabtarhiggs?call> Xing
<https://www.xing.com/profile/Fabien_Tarrade>

Fabien Tarrade

2016-Oct-02 16:51 UTC

head link

[R] remove a "corrupted file" after using download.file() with R on Windows 7

Hi Henrik,
> 1. It could be that a virus checker locks the file.This is some internal document so I don't think so and this was monitor 
by our IT security team.> 2. There are Windows software tools that identify which process locks
> a particular file, e.g. LockHunter (http://lockhunter.com/).  Those
> should help you figure out what's going on.
ok, thanks.> 3. R.utils::downloadFile() tries it's best to download files
> atomically, i.e. it either gives you a fully downloaded file or not
> all.  In your case, you might still end up with a temporary corrupt
> file, but at least it will have a filename that is different than the
> one you ask for.My issue is that if I run over 300'000 documents and I will get the same 
amount of corrupted files.
I will try some solution with R or try with python

Thanks
Cheers
Fabien>
>> Hi there,
>>
>> Sometime download.file() failed to download the file and I would like
to
>> remove the correspond file.
>> The issue is that I am not able to do it and Windows complain that the
file
>> is use by another application.
>> I try to closeAllConnections(), or unlink() before removing the file
but
>> without sucess.
>>
>> Any idea how I should proceed &
>>
>> Please find the code below
>>
>>   # consider warning as an error
>>    options(warn=2)
>>
>>    # try to download the file
>>    tryCatch({
>>      download.file(url,path_file,mode="wb",quiet=quiet)
>>      return(0)
>>    },error = function(e){
>>      if(verbose){
>>        print(e)
>>        print(e$message)
>>      }
>>      # close file when it failed
>>      if (file.exists(path_file)){
>>        closeAllConnections()
>>        #unlink(path_file, recursive=TRUE)
>>        #file.create(path_file,overwrite=TRUE,showWarning=TRUE)
>>        #system(paste0('open "', path_file,
'"'))
>>        file.remove(path_file,overwrite=TRUE,showWarning=TRUE)
>>      }
>>      return(1)
>>      }
>> )
>>
>> Thanks a lot
>> Cheers
>> Fabien
>>
>> --
>> Dr Fabien Tarrade
>>
>> Quantitative Analyst/Developer - Data Scientist
>>
>> Senior data analyst specialised in the modelling, processing and
statistical
>> treatment of data.
>> PhD in Physics, 10 years of experience as researcher at the forefront
of
>> international scientific research.
>> Fascinated by finance and data modelling.
>>
>> Geneva, Switzerland
>>
>> Email : contact at fabien-tarrade.eu <mailto:contact at
fabien-tarrade.eu>
>> Phone : www.fabien-tarrade.eu <http://www.fabien-tarrade.eu>
>> Phone : +33 (0)6 14 78 70 90
>>
>> LinkedIn <http://ch.linkedin.com/in/fabientarrade/> Twitter
>> <https://twitter.com/fabtar> Google
>> <https://plus.google.com/+FabienTarradeProfile/posts> Facebook
>> <https://www.facebook.com/fabien.tarrade.eu> Google
<skype:fabtarhiggs?call>
>> Xing <https://www.xing.com/profile/Fabien_Tarrade>
>>
>> ______________________________________________
>> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
-- 
Dr Fabien Tarrade

Quantitative Analyst/Developer - Data Scientist

Senior data analyst specialised in the modelling, processing and 
statistical treatment of data.
PhD in Physics, 10 years of experience as researcher at the forefront of 
international scientific research.
Fascinated by finance and data modelling.

Geneva, Switzerland

Email : contact at fabien-tarrade.eu <mailto:contact at fabien-tarrade.eu>
Phone : www.fabien-tarrade.eu <http://www.fabien-tarrade.eu>
Phone : +33 (0)6 14 78 70 90

LinkedIn <http://ch.linkedin.com/in/fabientarrade/> Twitter 
<https://twitter.com/fabtar> Google 
<https://plus.google.com/+FabienTarradeProfile/posts> Facebook 
<https://www.facebook.com/fabien.tarrade.eu> Google 
<skype:fabtarhiggs?call> Xing
<https://www.xing.com/profile/Fabien_Tarrade>

R help - Sep 2016 - remove a "corrupted file" after using download.file() with R on Windows 7

[R] remove a "corrupted file" after using download.file() with R on Windows 7

[R] remove a "corrupted file" after using download.file() with R on Windows 7

[R] remove a "corrupted file" after using download.file() with R on Windows 7

[R] remove a "corrupted file" after using download.file() with R on Windows 7

[R] remove a "corrupted file" after using download.file() with R on Windows 7