thr3ads.net - asterisk users - [asterisk-users] Multiple readfile oddities, newlines etc [Oct 2016]

If this information is useful, please help other people find it:
Share via:

Jonathan H

2016-Oct-17 13:31 UTC

[asterisk-users] Multiple readfile oddities, newlines etc

I have a plain text file, ASCII, unix line breaks. 1 single line, and all
that is in it is the word "radio".

Here's some test dialplan:

exten => 5,1,Verbose(Context: ${CONTEXT} Exten:${EXTEN})

    same => n,Set(feature=${FILE(/home/test/feature-1.txt,0,1,l,u)})
    same => n,Verbose(${feature})

    same => n,Set(featurefile=/home/test/feature-1.txt)
    same => n,Set(feature2=${FILE(${featurefile},0,1,l,u)})
    same => n,Verbose(${feature2})

Both should output "radio", right? Here's the output:

    -- Executing [5 at fromvoipfone201:2] Set("PJSIP/6001-00000052",
"feature=radio") in new stack
    -- Executing [5 at fromvoipfone201:3]
Verbose("PJSIP/6001-00000052",
"radio") in new stack
radio

    -- Executing [5 at fromvoipfone201:4] Set("PJSIP/6001-00000052",
"featurefile=/home/test/feature-1.txt") in new stack
    -- Executing [5 at fromvoipfone201:5] Set("PJSIP/6001-00000052",
"feature2*radi*") in new stack
                                                        *GAAAAARRRGGGHHHH!
 ^^^^^*

    -- Executing [5 at fromvoipfone201:6]
Verbose("PJSIP/6001-00000052",
"radi") in new stack

And this is what's just at the top of the script. If I put it way down in
another context:

    -- Executing [s at track-handler:3] Verbose("Local/s at
root-00000026;2",
"/home/test/feature-1") in new stack
/home/test/feature-1
    -- Executing [s at track-handler:4] Set("Local/s at
root-00000026;2",
"feature=*radio??*") in new stack
                                                    *EVEN MORE**
GAAAAARRRGGGHHHH!
 ^^^^^*

[Oct 17 13:29:33] ERROR[5093][C-0000003c]: json.c:704 ast_json_vpack: Error
building JSON from '{s: s, s: s}': Invalid UTF-8 string.
[Oct 17 13:29:33] ERROR[5093][C-0000003c]: stasis_channels.c:773
ast_channel_publish_varset: Error creating message
    -- Executing [s at track-handler:5] Verbose("Local/s at
root-00000026;2",
"radio??") in new stack
radio??
    -- Executing [s at track-handler:6] GotoIf("Local/s at
root-00000026;2",
"1?radio??,s,1") in new stack


So, at the top of the file, when it's just a straight file as the filename,
it gives the full word. If the filename is a variable, it strips and
character. And further down, same thing, but it adds two weird blocks,
which appear side by side in the console, but weirdly, stacked up in this
email.

(Oh, by the way, if there is more than one line in the file, even if I used
"u" for line breaks, it adds a newline to the variable, Is that
correct?)

What am I missing? I've opened it in both nano and notepad++, I've used
iconv and all the tools I can think of to check that file, and all the
asterisk conf files, too.
It all looks as it should here:

$ cat feature-1.txt
radio$ file feature-1.txt
feature-1.txt: ASCII text
$ wc -l feature-1.txt
0 feature-1.txt
wc -c feature-1.txt
5 feature-1.txt

After 6 hours struggling with this, I think I'm starting to lose the plot.
Can anyone tell me where I'm going wrong? Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.digium.com/pipermail/asterisk-users/attachments/20161017/392f13bd/attachment.html>

Sebastian Nielsen

2016-Oct-17 13:50 UTC

head link

[asterisk-users] Multiple readfile oddities, newlines etc

Theres always garbage in the end of the files.

 

I do this when I want to read a file:

    same => n,Set(featurefile=/home/test/feature-1.txt)
    same => n,Set(unfilteredfeat2=${FILE(${featurefile},0,1,l,u)})

    same => n,Set(feature2=${SHIFT(unfilteredfeat2)})

 

After that, add a , inside end of the file, so

 

Cat feature-1.txt

Reads:

radio,

 

Thus if there is garbage in the file, it will happen after the ,. Same if
garbage happens to find its way into the end of the variable for some reason.





Fr?n: asterisk-users-bounces at lists.digium.com [mailto:asterisk-users-bounces
at lists.digium.com] F?r Jonathan H
Skickat: den 17 oktober 2016 15:32
Till: Asterisk Users Mailing List - Non-Commercial Discussion <asterisk-users
at lists.digium.com>
?mne: [asterisk-users] Multiple readfile oddities, newlines etc

 

I have a plain text file, ASCII, unix line breaks. 1 single line, and all that
is in it is the word "radio".

Here's some test dialplan:

exten => 5,1,Verbose(Context: ${CONTEXT} Exten:${EXTEN})

    same => n,Set(feature=${FILE(/home/test/feature-1.txt,0,1,l,u)})
    same => n,Verbose(${feature})
   
    same => n,Set(featurefile=/home/test/feature-1.txt)
    same => n,Set(feature2=${FILE(${featurefile},0,1,l,u)})
    same => n,Verbose(${feature2})

Both should output "radio", right? Here's the output:

    -- Executing [5 at fromvoipfone201:2] Set("PJSIP/6001-00000052",
"feature=radio") in new stack
    -- Executing [5 at fromvoipfone201:3]
Verbose("PJSIP/6001-00000052", "radio") in new stack
radio

    -- Executing [5 at fromvoipfone201:4] Set("PJSIP/6001-00000052",
"featurefile=/home/test/feature-1.txt") in new stack
    -- Executing [5 at fromvoipfone201:5] Set("PJSIP/6001-00000052",
"feature2=radi") in new stack

                                                        GAAAAARRRGGGHHHH!  ^^^^^

 

    -- Executing [5 at fromvoipfone201:6]
Verbose("PJSIP/6001-00000052", "radi") in new stack

And this is what's just at the top of the script. If I put it way down in
another context:

    -- Executing [s at track-handler:3] Verbose("Local/s at
root-00000026;2", "/home/test/feature-1") in new stack
/home/test/feature-1
    -- Executing [s at track-handler:4] Set("Local/s at
root-00000026;2", "feature=radio??") in new stack

                                                    EVEN MORE GAAAAARRRGGGHHHH! 
^^^^^


[Oct 17 13:29:33] ERROR[5093][C-0000003c]: json.c:704 ast_json_vpack: Error
building JSON from '{s: s, s: s}': Invalid UTF-8 string.
[Oct 17 13:29:33] ERROR[5093][C-0000003c]: stasis_channels.c:773
ast_channel_publish_varset: Error creating message
    -- Executing [s at track-handler:5] Verbose("Local/s at
root-00000026;2", "radio??") in new stack
radio??
    -- Executing [s at track-handler:6] GotoIf("Local/s at
root-00000026;2", "1?radio??,s,1") in new stack

 

So, at the top of the file, when it's just a straight file as the filename,
it gives the full word. If the filename is a variable, it strips and character.
And further down, same thing, but it adds two weird blocks, which appear side by
side in the console, but weirdly, stacked up in this email.

 

(Oh, by the way, if there is more than one line in the file, even if I used
"u" for line breaks, it adds a newline to the variable, Is that
correct?)

What am I missing? I've opened it in both nano and notepad++, I've used
iconv and all the tools I can think of to check that file, and all the asterisk
conf files, too.
It all looks as it should here:


$ cat feature-1.txt
radio$ file feature-1.txt
feature-1.txt: ASCII text
$ wc -l feature-1.txt
0 feature-1.txt
wc -c feature-1.txt
5 feature-1.txt

 

After 6 hours struggling with this, I think I'm starting to lose the plot.
Can anyone tell me where I'm going wrong? Thanks.

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.digium.com/pipermail/asterisk-users/attachments/20161017/55fd86ad/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6298 bytes
Desc: S/MIME Cryptographic Signature
URL:
<http://lists.digium.com/pipermail/asterisk-users/attachments/20161017/55fd86ad/attachment.bin>

Jonathan H

2016-Oct-17 14:49 UTC

head link

[asterisk-users] Multiple readfile oddities, newlines etc

Just by chance, I was browsing the mailing list and it looks like
Sebastian's reply made it to the list, but not to me because it's
apparently blacklisted by UCEPROTECTL3.

This is what he wrote:
> Theres always garbage in the end of the files.
> I do this when I want to read a file:
>     same => n,Set(featurefile=/home/test/feature-1.txt)
>     same => n,Set(unfilteredfeat2=${FILE(${featurefile},0,1,l,u)})
>     same => n,Set(feature2=${SHIFT(unfilteredfeat2)})
> After that, add a , inside end of the file, so
> Cat feature-1.txt
> Reads:
> radio,
> Thus if there is garbage in the file, it will happen after the ,. Same if
garbage happens to find its way into the end of the variable for some reason.
Well, OK, that sort of works, but it revealed even more odd things.
And, if tools like wc are saying that a file is a certain length, how
can there be junk hiding in it?

So, then I found that having different lengths in the file messed
things up, too. Can anyone tell me this is normal?!?

Have a look, this is for changing the following words in the
features.txt. In all of the examples, the LEN function gets it wrong,
too.

radio < gets cut a letter short even with SHIFT trimmer
radio, < works, but only by chance because , makes it 6 characters
radui12345 < garbage at end
radio12345, < shift trimmer removes garbage
rad < junk at end


Context: fromvoipfone201 Exten:5
 Using a string, feature is set to ---radio--- and is 7 long
 Using a variable, feature2 is set to ---radi--- and is 8 long
 Using a string with shift method, feature3 is set to ---radi--- and is 8 long

Context: fromvoipfone201 Exten:5
 Using a string, feature is set to ---radio,--- and is 7 long
 Using a variable, feature2 is set to ---radio,--- and is 8 long
 Using a string with shift method, feature3 is set to ---radio--- and is 8 long

Context: fromvoipfone201 Exten:5
 Using a string, feature is set to ---radio12345--- and is 7 long
[Oct 17 15:30:29] ERROR[5813][C-00000073]: json.c:704 ast_json_vpack:
Error building JSON from '{s: s, s: s}': Invalid UTF-8 string.
[Oct 17 15:30:29] ERROR[5813][C-00000073]: stasis_channels.c:773
ast_channel_publish_varset: Error creating message
 Using a variable, feature2 is set to ---radio12345G?--- and is 8 long
[Oct 17 15:30:29] ERROR[5813][C-00000073]: json.c:704 ast_json_vpack:
Error building JSON from '{s: s, s: s}': Invalid UTF-8 string.
[Oct 17 15:30:29] ERROR[5813][C-00000073]: stasis_channels.c:773
ast_channel_publish_varset: Error creating message
[Oct 17 15:30:29] ERROR[5813][C-00000073]: json.c:704 ast_json_vpack:
Error building JSON from '{s: s, s: s}': Invalid UTF-8 string.
[Oct 17 15:30:29] ERROR[5813][C-00000073]: stasis_channels.c:773
ast_channel_publish_varset: Error creating message
 Using a string with shift method, feature3 is set to
---radio12345G?--- and is 8 long

Context: fromvoipfone201 Exten:5
 Using a string, feature is set to ---radio12345,--- and is 7 long
[Oct 17 15:30:35] ERROR[5814][C-00000074]: json.c:704 ast_json_vpack:
Error building JSON from '{s: s, s: s}': Invalid UTF-8 string.
[Oct 17 15:30:35] ERROR[5814][C-00000074]: stasis_channels.c:773
ast_channel_publish_varset: Error creating message
 Using a variable, feature2 is set to ---radio12345,?--- and is 8 long
[Oct 17 15:30:35] ERROR[5814][C-00000074]: json.c:704 ast_json_vpack:
Error building JSON from '{s: s, s: s}': Invalid UTF-8 string.
[Oct 17 15:30:35] ERROR[5814][C-00000074]: stasis_channels.c:773
ast_channel_publish_varset: Error creating message
[Oct 17 15:30:35] ERROR[5814][C-00000074]: json.c:704 ast_json_vpack:
Error building JSON from '{s: s, s: s}': Invalid UTF-8 string.
[Oct 17 15:30:35] ERROR[5814][C-00000074]: stasis_channels.c:773
ast_channel_publish_varset: Error creating message
 Using a string with shift method, feature3 is set to ---radio12345---
and is 8 long

Context: fromvoipfone201 Exten:5
 Using a string, feature is set to ---rad--- and is 7 long
[Oct 17 15:30:43] ERROR[5815][C-00000075]: json.c:704 ast_json_vpack:
Error building JSON from '{s: s, s: s}': Invalid UTF-8 string.
[Oct 17 15:30:43] ERROR[5815][C-00000075]: stasis_channels.c:773
ast_channel_publish_varset: Error creating message
 Using a variable, feature2 is set to ---rad?--- and is 8 long
[Oct 17 15:30:43] ERROR[5815][C-00000075]: json.c:704 ast_json_vpack:
Error building JSON from '{s: s, s: s}': Invalid UTF-8 string.
[Oct 17 15:30:43] ERROR[5815][C-00000075]: stasis_channels.c:773
ast_channel_publish_varset: Error creating message
[Oct 17 15:30:43] ERROR[5815][C-00000075]: json.c:704 ast_json_vpack:
Error building JSON from '{s: s, s: s}': Invalid UTF-8 string.
[Oct 17 15:30:43] ERROR[5815][C-00000075]: stasis_channels.c:773
ast_channel_publish_varset: Error creating message
 Using a string with shift method, feature3 is set to ---rad?--- and is 8 long

So, using the SHIFT trimmer cuts the junk, but still loses the last
character in some cases.

Here's the dialplan

    same => n,Set(feature=${FILE(/home/test/feature-1.txt,0,1,l,u)})
    same => n,Verbose(1,Using a string, feature is set to
---${feature}--- and is ${LEN(feature)} long)

    same => n,Set(featurefile=/home/test/feature-1.txt)
    same => n,Set(feature2=${FILE(${featurefile},0,1,l,u)})
    same => n,Verbose(1,Using a variable, feature2 is set to
---${feature2}--- and is ${LEN(feature2)} long)

    same => n,Set(unfilteredfeat=${FILE(${featurefile},0,1,l,u)})
    same => n,Set(feature3=${SHIFT(unfilteredfeat)})
    same => n,Verbose(1,Using a string with shift method, feature3 is
set to ---${feature3}--- and is ${LEN(feature3)} long)

Bug or... "feature"?!

On 17 October 2016 at 14:31, Jonathan H <lardconcepts at gmail.com>
wrote:> I have a plain text file, ASCII, unix line breaks. 1 single line, and all
> that is in it is the word "radio".
>
> Here's some test dialplan:
>
> exten => 5,1,Verbose(Context: ${CONTEXT} Exten:${EXTEN})
>
>     same => n,Set(feature=${FILE(/home/test/feature-1.txt,0,1,l,u)})
>     same => n,Verbose(${feature})
>
>     same => n,Set(featurefile=/home/test/feature-1.txt)
>     same => n,Set(feature2=${FILE(${featurefile},0,1,l,u)})
>     same => n,Verbose(${feature2})
>
> Both should output "radio", right? Here's the output:
>
>     -- Executing [5 at fromvoipfone201:2]
Set("PJSIP/6001-00000052",
> "feature=radio") in new stack
>     -- Executing [5 at fromvoipfone201:3]
Verbose("PJSIP/6001-00000052",
> "radio") in new stack
> radio
>
>     -- Executing [5 at fromvoipfone201:4]
Set("PJSIP/6001-00000052",
> "featurefile=/home/test/feature-1.txt") in new stack
>     -- Executing [5 at fromvoipfone201:5]
Set("PJSIP/6001-00000052",
> "feature2=radi") in new stack
>                                                         GAAAAARRRGGGHHHH!
> ^^^^^
>
>     -- Executing [5 at fromvoipfone201:6]
Verbose("PJSIP/6001-00000052",
> "radi") in new stack
>
> And this is what's just at the top of the script. If I put it way down
in
> another context:
>
>     -- Executing [s at track-handler:3] Verbose("Local/s at
root-00000026;2",
> "/home/test/feature-1") in new stack
> /home/test/feature-1
>     -- Executing [s at track-handler:4] Set("Local/s at
root-00000026;2",
> "feature=radio??") in new stack
>                                                     EVEN MORE
> GAAAAARRRGGGHHHH!  ^^^^^
>
> [Oct 17 13:29:33] ERROR[5093][C-0000003c]: json.c:704 ast_json_vpack: Error
> building JSON from '{s: s, s: s}': Invalid UTF-8 string.
> [Oct 17 13:29:33] ERROR[5093][C-0000003c]: stasis_channels.c:773
> ast_channel_publish_varset: Error creating message
>     -- Executing [s at track-handler:5] Verbose("Local/s at
root-00000026;2",
> "radio??") in new stack
> radio??
>     -- Executing [s at track-handler:6] GotoIf("Local/s at
root-00000026;2",
> "1?radio??,s,1") in new stack
>
>
> So, at the top of the file, when it's just a straight file as the
filename,
> it gives the full word. If the filename is a variable, it strips and
> character. And further down, same thing, but it adds two weird blocks,
which
> appear side by side in the console, but weirdly, stacked up in this email.
>
> (Oh, by the way, if there is more than one line in the file, even if I used
> "u" for line breaks, it adds a newline to the variable, Is that
correct?)
>
> What am I missing? I've opened it in both nano and notepad++, I've
used
> iconv and all the tools I can think of to check that file, and all the
> asterisk conf files, too.
> It all looks as it should here:
>
> $ cat feature-1.txt
> radio$ file feature-1.txt
> feature-1.txt: ASCII text
> $ wc -l feature-1.txt
> 0 feature-1.txt
> wc -c feature-1.txt
> 5 feature-1.txt
>
> After 6 hours struggling with this, I think I'm starting to lose the
plot.
> Can anyone tell me where I'm going wrong? Thanks.

Pete Mundy

2016-Oct-17 21:10 UTC

head link

[asterisk-users] Multiple readfile oddities, newlines etc

> On 18/10/2016, at 2:31 am, Jonathan H <lardconcepts at gmail.com>
wrote:
> 
> I have a plain text file, ASCII, unix line breaks. 1 single line, and all
that is in it is the word "radio".
> 
> <snip>
Heya Jonathan

Interesting problem!

Unfortunately I can't help with suitable dialplan code to resolve your
issue, but I thought I'd at least throw this snippet out to you:

If you want to know what is _really_ in that file (including all invisible
characters and anything else that wc etc might not count), pipe it through
'hexdump'. Then you get to see the raw contents of the file in hex and
can compare it to other files or the output you get from other tools (and you
can look up the ASCII char codes for invisible chars to explore what they are
etc).

Eg:

	cat /home/test/feature-1.txt | hexdump

Hope this tool helps you in your quest :)

Pete

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.digium.com/pipermail/asterisk-users/attachments/20161018/ca9d96ed/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3577 bytes
Desc: not available
URL:
<http://lists.digium.com/pipermail/asterisk-users/attachments/20161018/ca9d96ed/attachment.bin>

Steve Edwards

2016-Oct-17 21:38 UTC

head link

[asterisk-users] Multiple readfile oddities, newlines etc

On Tue, 18 Oct 2016, Pete Mundy wrote:
> If you want to know what is _really_ in that file (including all 
> invisible characters and anything else that wc etc might not count), 
> pipe it through 'hexdump'.
> 
> cat?/home/test/feature-1.txt | hexdump
Or just:

 	hexdump /home/test/feature-1.txt

-- 
Thanks in advance,
-------------------------------------------------------------------------
Steve Edwards       sedwards at sedwards.com      Voice: +1-760-468-3867 PST
             https://www.linkedin.com/in/steve-edwards-4244281

Possibly Parallel Threads

Search for more reasonably related threads

asterisk users - Oct 2016 - Multiple readfile oddities, newlines etc

[asterisk-users] Multiple readfile oddities, newlines etc

[asterisk-users] Multiple readfile oddities, newlines etc

[asterisk-users] Multiple readfile oddities, newlines etc

[asterisk-users] Multiple readfile oddities, newlines etc

[asterisk-users] Multiple readfile oddities, newlines etc

Possibly Parallel Threads