How to replace spaces with in Square brackets using sed command

mosabama

New Member
Joined
Jul 4, 2022
Messages
8
Reaction score
0
Credits
63
Hello,

I am trying to replace spaces in brackes with ","

for example:
"NIST 800-171A": ["3.11.2[a] 3.11.2 3.11.2[c] 3.11.2[d] 3.11.2[e] 3.11.3[a] 3.11.3"],
Will be:
"NIST 800-171A": ["3.11.2[a]","3.11.2","3.11.2[c]","3.11.2[d]","3.11.2[e]","3.11.3[a]","3.11.3"],

There are other spaces in the lines before and after the closed brackets, but I do not want other spaces in the line to be effected. and if the brackets do not contain spaces then nothing changes

Thank you for your help in advance :)
 


OP
M

mosabama

New Member
Joined
Jul 4, 2022
Messages
8
Reaction score
0
Credits
63
I tried these:
sed -e :1 -e 's/\(\[[^]]*\)[[:space:]]/\",\"/g;t1'
sed '/\[/,/]/{s/[ /]/\",\"/g}'
sed '/NIST.*:/s/\ /",\"/g'

but they seemed to replace all spaces regardless where they are in the line.
 
Last edited:

SlowCoder

Gold Member
Gold Supporter
Joined
May 2, 2022
Messages
456
Reaction score
314
Credits
3,611
Code:
sed -e 's/ /,/g' -e 's/NIST,/NIST /' -e 's/:,\[/: \['/

This assumes all input follows the same format, and starts with NIST.
 
OP
M

mosabama

New Member
Joined
Jul 4, 2022
Messages
8
Reaction score
0
Credits
63
Thank you,
this seems to be working to some extent, but also its adding the comma in all spaces, not only between the brackets:

example output:
,,,,,,,,,,,,"NIST 800-161,rev,1,Level,2": ["SI-20"],
,,,,,,,,,,,,"NIST 800-161,rev,1,Level,3": ["SI-20"],
,,,,,,,,,,,,"NIST Privacy,Framework,v1.0": ["PR.PO-P10"],
,,,,,,,,,,,,"NIST SSDF": ["RV.1,RV.1.1,RV.1.2,RV.1.3,RV.3,RV.3.1,RV.3.2"],
,,,,,,,,,,,,"NIST 800-53,rev4": ["SI-2,SI-3(2)"],
 

SlowCoder

Gold Member
Gold Supporter
Joined
May 2, 2022
Messages
456
Reaction score
314
Credits
3,611
I suspect you're going to be looking at a combo sed and awk solution. I tried poking around to get it all to work, but ultimately failed.
 
OP
M

mosabama

New Member
Joined
Jul 4, 2022
Messages
8
Reaction score
0
Credits
63
Another approach would be replacing any spaces after the column, space and an opening bracket. ": [", if any space is found after these 3 characters it would be changed to ","

not sure if this can help find a solution using sed.
 

SlowCoder

Gold Member
Gold Supporter
Joined
May 2, 2022
Messages
456
Reaction score
314
Credits
3,611
Another approach would be replacing any spaces after the column, space and an opening bracket. ": [", if any space is found after these 3 characters it would be changed to ","

not sure if this can help find a solution using sed.
That was essentially the solution I was going for, which I believe can mostly be done with awk.

Not as elegant, but you might be able to build a looped bash script to split the part before ": [" from the part you want to operate on, run your sed, then reassemble and store.
 

SlowCoder

Gold Member
Gold Supporter
Joined
May 2, 2022
Messages
456
Reaction score
314
Credits
3,611
Alright, you got me on a learning roll, so I've been digging in and learning a bit more about awk. I'm using the following dataset:
Code:
            "NIST 800-171A": ["3.11.2[a] 3.11.2 3.11.2[c] 3.11.2[d] 3.11.2[e] 3.11.3[a] 3.11.3"],
            "NIST 800-161,rev,1,Level,2": ["SI-20"],
            "NIST 800-161,rev,1,Level,3": ["SI-20"],
            "NIST Privacy,Framework,v1.0": ["PR.PO-P10"],
            "NIST SSDF": ["RV.1,RV.1.1,RV.1.2,RV.1.3,RV.3,RV.3.1,RV.3.2"],
            "NIST 800-53,rev4": ["SI-2,SI-3(2)"],

My latest code:
Code:
cat test | awk 'BEGIN {FS = ": \\["} ; {printf $1 ": ["} ; { "echo " $2 "| sed \"s/ /,/g\"" | getline x; print x }'

The output:
Code:
            "NIST 800-171A": [3.11.2[a],3.11.2,3.11.2[c],3.11.2[d],3.11.2[e],3.11.3[a],3.11.3],
            "NIST 800-161,rev,1,Level,2": [SI-20],
            "NIST 800-161,rev,1,Level,3": [SI-20],
            "NIST Privacy,Framework,v1.0": [PR.PO-P10],
            "NIST SSDF": [RV.1,RV.1.1,RV.1.2,RV.1.3,RV.3,RV.3.1,RV.3.2],
            "NIST 800-53,rev4": [SI-2,SI-3(2)],

Only one problem seems to remain; the getline command, which receives the sed conversion, doesn't preserve the quotes from $2.

If anyone has a simpler method, please speak out and teach us!
 
OP
M

mosabama

New Member
Joined
Jul 4, 2022
Messages
8
Reaction score
0
Credits
63
Wow, Thanks SlowCoder, I appreciate you trying to help me with this. I Will try that and play with it and see if I can do anything as well from my side <3
 
OP
M

mosabama

New Member
Joined
Jul 4, 2022
Messages
8
Reaction score
0
Credits
63
I am actually trying to replace the spaces with ","

This includes the quotes, I Am trying to add them with escape chars in the sed part of your code, but its would not add them still
 

SlowCoder

Gold Member
Gold Supporter
Joined
May 2, 2022
Messages
456
Reaction score
314
Credits
3,611
Code:
cat test | awk 'BEGIN {FS = ": \\["} ; {printf $1 ": ["} ; { "echo " $2 "| sed \"s/ /,/g\"" | getline x; print x }' | sed -e "s/: \[/: \[\"/" -e "s/\(.*\)\]/\1\"]/"
Added the extra sed statement at the end. First part adds the first quote, second part adds the last quote.

Output:
Code:
            "NIST 800-171A": ["3.11.2[a],3.11.2,3.11.2[c],3.11.2[d],3.11.2[e],3.11.3[a],3.11.3"],
            "NIST 800-161,rev,1,Level,2": ["SI-20"],
            "NIST 800-161,rev,1,Level,3": ["SI-20"],
            "NIST Privacy,Framework,v1.0": ["PR.PO-P10"],
            "NIST SSDF": ["RV.1,RV.1.1,RV.1.2,RV.1.3,RV.3,RV.3.1,RV.3.2"],
            "NIST 800-53,rev4": ["SI-2,SI-3(2)"],

Hopefully this does it for you? I don't think you can add the quotes via the internal sed, because getline strips the quotes after sed is complete. The two piped sed commands appended to the end look like a group of ASCII ravers.

Unfortunately, we are getting very format-specific, but as long as your data format is stable you should be ok?
 
OP
M

mosabama

New Member
Joined
Jul 4, 2022
Messages
8
Reaction score
0
Credits
63
IT does not seem that this is adding the quotes, and it kind of acts weird in other places,

For Example, in likes that looks like this:
"Risk R-EX-2": "R-EX-2"

it turns them into :
"Risk R-EX-2": "R-EX-2",: ["NFO-PS-6"],
I am not sure where that section came from!!!


Do you know if this is achievable in Visual code studio instead, would it be easier there, Or even Vim. I am not stuck to sed or Awk as long as there is a solution somewhere else.

Thank you
 

SlowCoder

Gold Member
Gold Supporter
Joined
May 2, 2022
Messages
456
Reaction score
314
Credits
3,611
Do you know if this is achievable in Visual code studio instead, would it be easier there, Or even Vim. I am not stuck to sed or Awk as long as there is a solution somewhere else.
You requested a solution using sed, so I came as close to that as possible. Just about any language has the ability to work with data files and patterns, so feel free to use VS Code if you're more comfortable with it.

I think you need to determine all of the possible patterns your input data can take. Then you can determine what your delimiters can be, and massage your data from there. Until you do this, no solution can be 100% reliable.

Good luck!
 
$100 Digital Ocean Credit
Get a free VM to test out Linux!

Linux.org Hosting Donations
Consider making a donation


Top