CentOS - Jul 2020 - Off Topic bash question

I have a simple script:
#!/bin/bash
#
index=0
total=0
names=()
ip=()
while read -r LINE
do
NODENAME=`   echo $LINE | cut -f 1 -d ','`
IP=`         echo $LINE | cut -f 2 -d ','`
names[index]="$NODENAME"
ip[index]="$IP"
index=`expr index+1`
total=`expr total+1`
done <<< $(cat list.txt)

simple file:
more list.txt
name1,ip1
name2,ip2
name3,ip3

output when running:
 sh -x ./test_bash.sh
+ index=0
+ total=0
+ names=()
+ ip=()
++ cat list.txt
+ read -r LINE
++ echo name1,ip1 name2,ip2 name3,ip3
++ cut -f 1 -d ,
+ NODENAME=name1
++ echo name1,ip1 name2,ip2 name3,ip3
++ cut -f 2 -d ,
+ IP='ip1 name2'
+ names[index]=name1
+ ip[index]='ip1 name2'
++ expr index+1
+ index=index+1
++ expr total+1
+ total=total+1
+ read -r LINE
+ echo name1
name1


Question is why is it not reading one line at a time ?
All I get is the first one.
I'm just trying to build the array of the items in the file and then list
them at this point.

Thanks

Jerry

On 23/07/2020 15:46, Jerry Geis wrote:

Hi Jerry,

See below, inline, for some comments.
> I have a simple script:
> #!/bin/bash
> #
> index=0
> total=0
> names=()
> ip=()
> while read -r LINE
> do
> NODENAME=`   echo $LINE | cut -f 1 -d ','`
NODENAME=$(cut -d, -f1 <<< $LINE)

Notes: use $( instead of backticks. There's no need to quote the comma. 
Write less and write concisely.
> IP=`         echo $LINE | cut -f 2 -d ','`
IP=$(cut -d, -f2 <<< $LINE)
> names[index]="$NODENAME"
> ip[index]="$IP"
ip[$((index++))]="$IP"

This allows you to use the variable index, and then increment it by one 
for the next cycle of the loop.
> index=`expr index+1`
Not needed because of the post-increment (++) above.
> total=`expr total+1`
((total++))
> done <<< $(cat list.txt)
done < list.txt

The "<<<" operator does all manner of expansion of the input
and
supplies it as a single line. That not what you want. Just redirect 
stdin from the file instead.
> simple file:
> more list.txt
> name1,ip1
> name2,ip2
> name3,ip3
> 
> output when running:
>   sh -x ./test_bash.sh
> + index=0
> + total=0
> + names=()
> + ip=()
> ++ cat list.txt
> + read -r LINE
> ++ echo name1,ip1 name2,ip2 name3,ip3
This is happening because of the <<< operator.
> ++ cut -f 1 -d ,
> + NODENAME=name1
> ++ echo name1,ip1 name2,ip2 name3,ip3
> ++ cut -f 2 -d ,
> + IP='ip1 name2'
> + names[index]=name1
> + ip[index]='ip1 name2'
> ++ expr index+1
> + index=index+1
> ++ expr total+1
> + total=total+1
> + read -r LINE
> + echo name1
> name1
> 
> 
> Question is why is it not reading one line at a time ?
> All I get is the first one.
> I'm just trying to build the array of the items in the file and then
list
> them at this point.

Hi,

Some remarks:
- just try ? expr ${VAR} + 1 ? (with blanc between car)
- use simple redirection (? < ?) with  a simple reference to file (? done
< list.txt ?)
- use ? ${VAR} ? to manage variables
- you can use numeric notation to increment VAR (p.e. ? total=$(( ${total} + 1
)) ?)

An other way to do that: AWK.

Regards
> Le 23 juil. 2020 ? 15:46, Jerry Geis <jerry.geis at gmail.com> a
?crit :
> 
> I have a simple script:
> #!/bin/bash
> #
> index=0
> total=0
> names=()
> ip=()
> while read -r LINE
> do
> NODENAME=`   echo $LINE | cut -f 1 -d ','`
> IP=`         echo $LINE | cut -f 2 -d ','`
> names[index]="$NODENAME"
> ip[index]="$IP"
> index=`expr index+1`
> total=`expr total+1`
> done <<< $(cat list.txt)
> 
> simple file:
> more list.txt
> name1,ip1
> name2,ip2
> name3,ip3
> 
> output when running:
> sh -x ./test_bash.sh
> + index=0
> + total=0
> + names=()
> + ip=()
> ++ cat list.txt
> + read -r LINE
> ++ echo name1,ip1 name2,ip2 name3,ip3
> ++ cut -f 1 -d ,
> + NODENAME=name1
> ++ echo name1,ip1 name2,ip2 name3,ip3
> ++ cut -f 2 -d ,
> + IP='ip1 name2'
> + names[index]=name1
> + ip[index]='ip1 name2'
> ++ expr index+1
> + index=index+1
> ++ expr total+1
> + total=total+1
> + read -r LINE
> + echo name1
> name1
> 
> 
> Question is why is it not reading one line at a time ?
> All I get is the first one.
> I'm just trying to build the array of the items in the file and then
list
> them at this point.
> 
> Thanks
> 
> Jerry
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> https://lists.centos.org/mailman/listinfo/centos
--
Pierre Malard
Responsable architectures syst?me GeoSUD
IRD - UMR Espace-Dev - UMS CPST
Maison de la T?l?d?tection
500 rue Jean-Fran?ois Breton
34093 Montpellier Cx 5
France

	? L'utopie, c'est la v?rit? de demain ?
                                 Victor Hugo (1802-1885)
   |\      _,,,---,,_
   /,`.-'`'    -.  ;-;;,_
  |,4-  ) )-,_. ,\ (  `'-'
 '---''(_/--'  `-'\_)   ?r

perl -e '$_=q#: 3|\ 5_,3-3,2_:
3/,`.'"'"'`'"'"' 5-.  ;-;;,_:  |,A- 
) )-,_. ,\ (  `'"'"'-'"'"':
'"'"'-3'"'"'2(_/--'"'"'
`-'"'"'\_):
24?r::#;y#:#\n#;s#(\D)(\d+)#$1x$2#ge;print'
- --> Ce message n?engage que son auteur <--

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL:
<http://lists.centos.org/pipermail/centos/attachments/20200723/dd4fbfdf/attachment.sig>

Thanks, when I change it do the following I get a syntax error

#!/bin/bash
#
while read LINE
do
         echo $LINE
done < cat list.txt

./test_bash.sh
./test_bash.sh: line 6: syntax error near unexpected token `list.txt'
./test_bash.sh: line 6: ` done < cat list.txt'

On 23/07/2020 15:46, Jerry Geis wrote:

Hi Jerry,

You can do even better:

index=0
total=0
names=()
ip=()

IFS=,

while read -r NODENAME IP
do
   names[$index]="$NODENAME"
   ip[$((index++))]="$IP"
   ((total++))
done < list.txt

In this example, you set the input field separator (IFS) to the comma, 
so that the shell does the word splitting for you, and then allow the 
"read" call to assign to both variables in a single call. Now you
don't
need any "cut" calls. In fact, this example avoids forks completely,
by
just using native features of bash.

For small inputs, it doesn't matter, but if you were processing a large 
file, and your script was forking for each call to "cut" and creating
a
new process, you would incur a horrible amount of overhead for no reason.

Regards,
Anand
> I have a simple script:
> #!/bin/bash
> #
> index=0
> total=0
> names=()
> ip=()
> while read -r LINE
> do
> NODENAME=`   echo $LINE | cut -f 1 -d ','`
> IP=`         echo $LINE | cut -f 2 -d ','`
> names[index]="$NODENAME"
> ip[index]="$IP"
> index=`expr index+1`
> total=`expr total+1`
> done <<< $(cat list.txt)
> 
> simple file:
> more list.txt
> name1,ip1
> name2,ip2
> name3,ip3
> 
> output when running:
>   sh -x ./test_bash.sh
> + index=0
> + total=0
> + names=()
> + ip=()
> ++ cat list.txt
> + read -r LINE
> ++ echo name1,ip1 name2,ip2 name3,ip3
> ++ cut -f 1 -d ,
> + NODENAME=name1
> ++ echo name1,ip1 name2,ip2 name3,ip3
> ++ cut -f 2 -d ,
> + IP='ip1 name2'
> + names[index]=name1
> + ip[index]='ip1 name2'
> ++ expr index+1
> + index=index+1
> ++ expr total+1
> + total=total+1
> + read -r LINE
> + echo name1
> name1
> 
> 
> Question is why is it not reading one line at a time ?
> All I get is the first one.
> I'm just trying to build the array of the items in the file and then
list
> them at this point.
> 
> Thanks
> 
> Jerry
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> https://lists.centos.org/mailman/listinfo/centos
>

On Thu, Jul 23, 2020 at 4:25 PM Anand Buddhdev <anandb at ripe.net> wrote:
> On 23/07/2020 15:46, Jerry Geis wrote:
>
> Hi Jerry,
>
> See below, inline, for some comments.
>
>
> > while read -r LINE
> > do
> > NODENAME=`   echo $LINE | cut -f 1 -d ','`
>
> NODENAME=$(cut -d, -f1 <<< $LINE)
>
> Notes: use $( instead of backticks. There's no need to quote the comma.
> Write less and write concisely.
>
> > IP=`         echo $LINE | cut -f 2 -d ','`
>
> IP=$(cut -d, -f2 <<< $LINE)
>
> > names[index]="$NODENAME"
> > ip[index]="$IP"
>
>
I would add these considerations:
1) -r means to not consider \ character as an escape character so possibly
is not important to use it in your scenario
2) read support multiple variables parsing
3) read word separation (default space) can be modified using IFS (internal
field separator)

So I would change the script this way, together with other considerations

#!/bin/bash
#
IFS=','

index=0
total=0
names=()
ip=()
#while read -r LINE
while read NODENAME IP
do
#NODENAME=$(echo $LINE | cut -f 1 -d ',')
#IP=$(echo $LINE | cut -f 2 -d ',')
names[$((index++))]="$NODENAME"
ip[$((index++))]="$IP"
((total))
done < /tmp/list.txt

to verify you can add something like:

for i in ${names[@]} ${ip[@]}
do
echo $i
done

HIH,
Gianluca