freebsd stable - Jul 2016 - sed command does not behave equal from 10.3 to 11.0

are you sure you aren't hitting a port or something?

#  uname -r ; echo "abc_ABC.def" | sed -e 's/[^A-Z0-9]//g' ;
which sed ;
md5 /usr/bin/sed
10.3-STABLE
ABC
/usr/bin/sed
MD5 (/usr/bin/sed) = 34e6aedf3b42cbd6dd8379342626e0db


#  uname -r ; echo "abc_ABC.def" | sed -e 's/[^A-Z0-9]//g' ;
which sed ;
md5 /usr/bin/sed
11.0-BETA2
ABC
/usr/bin/sed
MD5 (/usr/bin/sed) = d3fddd6bcca17fc597d7c4598c3311d1






On 27 July 2016 at 09:20, Jos? Garc?a Juanino <jjuanino at gmail.com>
wrote:
> Hi FreeBSD stable,
>
> After upgrade to 11.0-BETA2, a lot of sed scripts are no longer valid:
>
> FreeBSD 10.3
> ---------------------
> $ echo "abc_ABC.def" | sed -e 's/[^A-Z0-9]//g'
> ABC
>
>
> FreeBSD 11.0
> --------------------
> $ echo "abc_ABC.def" | sed -e 's/[^A-Z0-9]//g'
> bcABCdef
>
>
> Indeed, in 11.0 you need to re rewrite the sentence as follows:
>
> $ echo "abc_ABC.def" | sed -e
's/[^[:upper:][:digit:]]//g'
> ABC
>
> In linux, sed behaves exactly as 10.3.
>
> Is this behaviour expected or is a bug?
>
> Best regards
> _______________________________________________
> freebsd-stable at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscribe at
freebsd.org"
>

On Wed, Jul 27, 2016 at 09:45:23AM +0100 I heard the voice of
krad, and lo! it spake thus:
> are you sure you aren't hitting a port or something?
Locale dependant.

% echo "abc_ABC.def" | env LANG=C sed -e 's/[^A-Z0-9]//g'
ABC

% echo "abc_ABC.def" | env LANG=en_US.UTF-8 sed -e
's/[^A-Z0-9]//g'
bcABCdef

(pre-branch -CURRENT)


-- 
Matthew Fuller     (MF4839)   |  fullermd at over-yonder.net
Systems/Network Administrator |  http://www.over-yonder.net/~fullermd/
           On the Internet, nobody can hear you scream.

On 27 July 2016 at 10:45, krad <kraduk at gmail.com>
wrote:
> are you sure you aren't hitting a port or something?
>
> #  uname -r ; echo "abc_ABC.def" | sed -e
's/[^A-Z0-9]//g' ; which sed ; md5
> /usr/bin/sed
> 10.3-STABLE
> ABC
> /usr/bin/sed
> MD5 (/usr/bin/sed) = 34e6aedf3b42cbd6dd8379342626e0db
In 10.3 I get the following:

$ freebsd-version -ku
$ which sed ; md5 /usr/bin/sed
/usr/bin/sed
MD5 (/usr/bin/sed) = 858696b78b6d8ed26a1e9835b1bcd89f


In 11.0-BETA2 I get:

$ freebsd-version -ku
$ which sed ; md5 /usr/bin/sed
/usr/bin/sed
MD5 (/usr/bin/sed) = 229d361061fcf528be12d513ad9a2cf3



I think that the issue is locale or encoding related, as it happens
also with gsed, but *not* with LANG=C:

 $ locale
LANG=es_ES.ISO8859-15
LC_CTYPE="es_ES.ISO8859-15"
LC_COLLATE="es_ES.ISO8859-15"
LC_TIME="es_ES.ISO8859-15"
LC_NUMERIC="es_ES.ISO8859-15"
LC_MONETARY="es_ES.ISO8859-15"
LC_MESSAGES="es_ES.ISO8859-15"
LC_ALL
$ echo "abc_ABC.def" | sed -e 's/[^A-Z0-9]//g'
bcABCdef ### wrong


$ export LANG=C
$  locale
LANG=C
LC_CTYPE="C"
LC_COLLATE="C"
LC_TIME="C"
LC_NUMERIC="C"
LC_MONETARY="C"
LC_MESSAGES="C"
LC_ALL
$ echo "abc_ABC.def" | sed -e 's/[^A-Z0-9]//g'
ABC  ##  right