Index out of range error when censoring cyrillic language #25

New issue

Closed

opened 2022-03-07 11:14:02 -05:00 by gdgcentrica · 2 comments

gdgcentrica commented

2022-03-07 11:14:02 -05:00

(Migrated from github.com)

Describe the bug

When i try to censor a word in cyrillic it throw a panic error

What do you see?

panic: runtime error: index out of range [8] with length 8

goroutine 62 [running]:

github.com/TwiN/go-away.(*ProfanityDetector).Censor(0xc00039bd28, {0xc0001a0c10, 0x10})
        /home/user/.gvm/pkgsets/go1.18rc1/global/pkg/mod/github.com/!twi!n/go-away@v1.6.0/goaway.go:179 +0x6d1

What do you expect to see?

A correct censorship of the word

List the steps that must be taken to reproduce this issue

filter := goaway.NewProfanityDetector().WithCustomDictionary(profanities, fp, fn)

cleanedWord := f.Censor("крывавы")

Version

1.6.0

Additional information

No response

### Describe the bug When i try to censor a word in cyrillic it throw a panic error ### What do you see? ```go panic: runtime error: index out of range [8] with length 8 goroutine 62 [running]: github.com/TwiN/go-away.(*ProfanityDetector).Censor(0xc00039bd28, {0xc0001a0c10, 0x10}) /home/user/.gvm/pkgsets/go1.18rc1/global/pkg/mod/github.com/!twi!n/go-away@v1.6.0/goaway.go:179 +0x6d1 ``` ### What do you expect to see? A correct censorship of the word ### List the steps that must be taken to reproduce this issue ```go filter := goaway.NewProfanityDetector().WithCustomDictionary(profanities, fp, fn) cleanedWord := f.Censor("крывавы") ``` ### Version 1.6.0 ### Additional information _No response_

👍 1

TwiN commented

2022-03-10 19:05:51 -05:00

(Migrated from github.com)

I think the problem is that there are 7 characters, but the len("крывавы") probably returns 14, so character replacement might fail around there:
TwiN/go-away@bd63346fc7/goaway.go (L178-L180)

I think the problem is that there are 7 characters, but the `len("крывавы")` probably returns 14, so character replacement might fail around there: https://github.com/TwiN/go-away/blob/bd63346fc7f0b3904d8211ed4215f3478e348fcd/goaway.go#L178-L180

martinhrvn commented

2022-03-15 11:23:55 -04:00

(Migrated from github.com)

I think I have a fix for that. It is true that instead of checking len(word) we should check len([]rune(word)) but also when doing rememberOriginalIndexes we should iterate []rune instead of string. I'll create PR shortly, as I can reproduce the issue even without cyrilic characters simply by using “ in the string before the profanity occurence

nope, it's not going to be that easy..

~I think I have a fix for that. It is true that instead of checking `len(word)` we should check `len([]rune(word))` but also when doing `rememberOriginalIndexes` we should iterate `[]rune` instead of `string`. I'll create PR shortly, as I can reproduce the issue even without cyrilic characters simply by using `“` in the string before the profanity occurence~ nope, it's not going to be that easy..

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

TwiN/go-away#25

No description provided.

Rows
Columns