CentOS - Dec 2014 - Asymmetric encryption for very large tar file

Am 17.12.2014 um 18:42 schrieb Les Mikesell <lesmikesell at
gmail.com>:
> On Wed, Dec 17, 2014 at 11:14 AM, Xinhuan Zheng
> <xzheng at christianbook.com> wrote:
>> I have a requirement that I need to use encryption technology to
encrypt
>> very large tar file on a daily basis. The tar file is over 250G size
and
>> those are data backup. Every night the server generated a 250G data
backup
>> and it?s tar?ed into one tarball file. I want to encrypt this big
tarball
>> file. So far I have tried two technologies with no success.
>> 1) generating RSA 2048 public/private key pair via ?openssl req -x509
>> -nodes -newkey rsa:2048 -keyout private.pem -out public.pem? command
and
>> uses the public key to encrypt the big tar file. The encryption command
I
>> used is "openssl smime -encrypt -aes256 -in  backup.tar -binary
-outform
>> DEM -out backup.tar.ssl  public.pem?. The resulting backup.tar.ssl file
is
>> only 2G then encryption process stops there and refuse to do more.
Cannot
>> get around 2G.
> 
> What happens if you use a pipeline or redirection instead of the -in
> and -out files?   I regularly write large tapes with something like:
> openssl aes-256-cbc -salt -k password <input.tar.gz  |dd bs=10240
> obs=10240 of=/dev/nst0
> Not quite the same, but there does not seem to be an inherent size
> limit in openssl as long as it is not handling files and it happens at
> a reasonable speed so it must be using the intel hardware support.


Furthermore - is there the need to use "one" big tar file? Despite 
having a capable workstation/server handling such big files, it has 
also advantages splitting such backups (e.g. man split) ...

--
LF

On 17/12/14 18:54, Leon Fauster wrote:
> Am 17.12.2014 um 18:42 schrieb Les Mikesell <lesmikesell at
gmail.com>:
>> On Wed, Dec 17, 2014 at 11:14 AM, Xinhuan Zheng
>> <xzheng at christianbook.com> wrote:
>>> I have a requirement that I need to use encryption technology to
encrypt
>>> very large tar file on a daily basis. The tar file is over 250G
size and
>>> those are data backup. Every night the server generated a 250G data
backup
>>> and it?s tar?ed into one tarball file. I want to encrypt this big
tarball
>>> file. So far I have tried two technologies with no success.
>>> 1) generating RSA 2048 public/private key pair via ?openssl req
-x509
>>> -nodes -newkey rsa:2048 -keyout private.pem -out public.pem?
command and
>>> uses the public key to encrypt the big tar file. The encryption
command I
>>> used is "openssl smime -encrypt -aes256 -in  backup.tar
-binary -outform
>>> DEM -out backup.tar.ssl  public.pem?. The resulting backup.tar.ssl
file is
>>> only 2G then encryption process stops there and refuse to do more.
Cannot
>>> get around 2G.
>>
>> What happens if you use a pipeline or redirection instead of the -in
>> and -out files?   I regularly write large tapes with something like:
>> openssl aes-256-cbc -salt -k password <input.tar.gz  |dd bs=10240
>> obs=10240 of=/dev/nst0
>> Not quite the same, but there does not seem to be an inherent size
>> limit in openssl as long as it is not handling files and it happens at
>> a reasonable speed so it must be using the intel hardware support.
> 
> 
> 
> Furthermore - is there the need to use "one" big tar file?
Despite
> having a capable workstation/server handling such big files, it has 
> also advantages splitting such backups (e.g. man split) ...
> 
> --
> LF
> 
> 
>  
> 
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> http://lists.centos.org/mailman/listinfo/centos
> 
Is it possible for you to use gpg? You could do something like:
tar zcf /something - | gpg -e -r otherkey | cat - > backup.tgz

Regards

On Wed, Dec 17, 2014 at 06:58:40PM +0100, Markus wrote:
> On 17/12/14 18:54, Leon Fauster wrote:
> > Am 17.12.2014 um 18:42 schrieb Les Mikesell <lesmikesell at
gmail.com>:
> >> On Wed, Dec 17, 2014 at 11:14 AM, Xinhuan Zheng
> >> <xzheng at christianbook.com> wrote:
> >>> I have a requirement that I need to use encryption technology
to encrypt
> >>> very large tar file on a daily basis. The tar file is over
250G size and
> >>> those are data backup. Every night the server generated a 250G
data backup
> >>> and it?s tar?ed into one tarball file. I want to encrypt this
big tarball
> >>> file. So far I have tried two technologies with no success.
> >>> 1) generating RSA 2048 public/private key pair via ?openssl
req -x509
> >>> -nodes -newkey rsa:2048 -keyout private.pem -out public.pem?
command and
> >>> uses the public key to encrypt the big tar file. The
encryption command I
> >>> used is "openssl smime -encrypt -aes256 -in  backup.tar
-binary -outform
> >>> DEM -out backup.tar.ssl  public.pem?. The resulting
backup.tar.ssl file is
> >>> only 2G then encryption process stops there and refuse to do
more. Cannot
> >>> get around 2G.
> >>
> >> What happens if you use a pipeline or redirection instead of the
-in
> >> and -out files?   I regularly write large tapes with something
like:
> >> openssl aes-256-cbc -salt -k password <input.tar.gz  |dd
bs=10240
> >> obs=10240 of=/dev/nst0
> >> Not quite the same, but there does not seem to be an inherent size
> >> limit in openssl as long as it is not handling files and it
happens at
> >> a reasonable speed so it must be using the intel hardware support.
> > 
> > 
> > 
> > Furthermore - is there the need to use "one" big tar file?
Despite
> > having a capable workstation/server handling such big files, it has 
> > also advantages splitting such backups (e.g. man split) ...
> > LF
<snip>
> 
> Is it possible for you to use gpg? You could do something like:
> tar zcf /something - | gpg -e -r otherkey | cat - > backup.tgz
> 
> Regards
or better yet:
tar zcf /something - | gpg -e -r otherkey > backup.tgz

If gpg can write to stdout, then it can write to a redirected stdout.
no need to use a superfluous cat in there.

and (without checking the man page) I'm not sure about the 'f' in
the
tar commandline... I thought the 'f' referred to the output file???

-- 
---- Fred Smith -- fredex at fcshome.stoneham.ma.us ----------------------------
                    Do you not know? Have you not heard? 
    The LORD is the everlasting God, the Creator of the ends of the earth. 
  He will not grow tired or weary, and his understanding no one can fathom.
----------------------------- Isaiah 40:28 (niv) -----------------------------

hi all

sorry my poor english..

but you need encrypt that large file in symetric way. use the asymetric way
( public/private key par) to encrypt the symetric key.
Em 17/12/2014 15:58, "Markus" <markus.scharitzer at gmail.com>
escreveu:
> On 17/12/14 18:54, Leon Fauster wrote:
> > Am 17.12.2014 um 18:42 schrieb Les Mikesell <lesmikesell at
gmail.com>:
> >> On Wed, Dec 17, 2014 at 11:14 AM, Xinhuan Zheng
> >> <xzheng at christianbook.com> wrote:
> >>> I have a requirement that I need to use encryption technology
to
> encrypt
> >>> very large tar file on a daily basis. The tar file is over
250G size
> and
> >>> those are data backup. Every night the server generated a 250G
data
> backup
> >>> and it?s tar?ed into one tarball file. I want to encrypt this
big
> tarball
> >>> file. So far I have tried two technologies with no success.
> >>> 1) generating RSA 2048 public/private key pair via ?openssl
req -x509
> >>> -nodes -newkey rsa:2048 -keyout private.pem -out public.pem?
command
> and
> >>> uses the public key to encrypt the big tar file. The
encryption
> command I
> >>> used is "openssl smime -encrypt -aes256 -in  backup.tar
-binary
> -outform
> >>> DEM -out backup.tar.ssl  public.pem?. The resulting
backup.tar.ssl
> file is
> >>> only 2G then encryption process stops there and refuse to do
more.
> Cannot
> >>> get around 2G.
> >>
> >> What happens if you use a pipeline or redirection instead of the
-in
> >> and -out files?   I regularly write large tapes with something
like:
> >> openssl aes-256-cbc -salt -k password <input.tar.gz  |dd
bs=10240
> >> obs=10240 of=/dev/nst0
> >> Not quite the same, but there does not seem to be an inherent size
> >> limit in openssl as long as it is not handling files and it
happens at
> >> a reasonable speed so it must be using the intel hardware support.
> >
> >
> >
> > Furthermore - is there the need to use "one" big tar file?
Despite
> > having a capable workstation/server handling such big files, it has
> > also advantages splitting such backups (e.g. man split) ...
> >
> > --
> > LF
> >
> >
> >
> >
> > _______________________________________________
> > CentOS mailing list
> > CentOS at centos.org
> > http://lists.centos.org/mailman/listinfo/centos
> >
>
> Is it possible for you to use gpg? You could do something like:
> tar zcf /something - | gpg -e -r otherkey | cat - > backup.tgz
>
> Regards
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> http://lists.centos.org/mailman/listinfo/centos
>