Collapse repeated characters (possibly using regex) #8

New issue

Open

opened 2020-10-09 12:45:50 -04:00 by finnbear · 1 comment

finnbear commented

2020-10-09 12:45:50 -04:00

(Migrated from github.com)

The program

package main

import (
	"fmt"
	"github.com/TwinProduction/go-away"
)

func main() {
	test("shit")
	test("shiit")
	test("shiiiiiit")
}

func test(word string) {
	fmt.Printf("\"%s\" profane? %t\n", word, goaway.IsProfane(word))
}

prints

"shit" profane? true
"shiit" profane? false
"shiiiiiit" profane? false

I suggest you include a list of regexes to match cases such as this.

An alternative would be to collapse all repeated characters before matching, although this would have an issue with any profanities that contain repeated characters that are no longer profane once collapsed.

The program ```go package main import ( "fmt" "github.com/TwinProduction/go-away" ) func main() { test("shit") test("shiit") test("shiiiiiit") } func test(word string) { fmt.Printf("\"%s\" profane? %t\n", word, goaway.IsProfane(word)) } ``` prints ``` "shit" profane? true "shiit" profane? false "shiiiiiit" profane? false ``` I suggest you include a list of regexes to match cases such as this. An alternative would be to collapse all repeated characters before matching, although this would have an issue with any profanities that contain repeated characters that are no longer profane once collapsed.

TwiN commented

2020-10-09 14:10:07 -04:00

(Migrated from github.com)

Completely agree with you on this. Tackling this is already in the list of TODOs on the README:

[TODO] All words that have the same character repeated more than twice in a row are removed (e.g. poooop -> poop)

NOTE: This is obviously not a perfect approach, as words like fuuck wouldn't be detected, but it's better than nothing.

I think one of the solutions could be to only allow specific characters to be repeated twice, such as o (e.g. poop) and t (e.g. letter), but collapse letters that aren't usually repeated into a single letter, like i (e.g. shiit) or u (e.g. fuuck).

I'm not sure regex is the best solution for this either, because Golang's Regex is known for not being the fastest, but it would be good to check what the performance difference is between multiple strings.Replace(...) and a single regex.

Anyways, thanks for creating an issue for this.
I can't promise when I'll be able to tackle the issue, but if you want to give it a try, you're welcome to :)

Completely agree with you on this. Tackling this is already in the list of TODOs on the README: >[TODO] All words that have the same character repeated more than twice in a row are removed (e.g. poooop -> poop) > - NOTE: This is obviously not a perfect approach, as words like fuuck wouldn't be detected, but it's better than nothing. I think one of the solutions could be to only allow specific characters to be repeated twice, such as `o` (e.g. `poop`) and `t` (e.g. letter), but collapse letters that aren't usually repeated into a single letter, like i (e.g. `shiit`) or u (e.g. `fuuck`). I'm not sure regex is the best solution for this either, because Golang's Regex is known for not being the fastest, but it would be good to check what the performance difference is between multiple `strings.Replace(...)` and a single regex. Anyways, thanks for creating an issue for this. I can't promise when I'll be able to tackle the issue, but if you want to give it a try, you're welcome to :)

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

TwiN/go-away#8

No description provided.

Rows
Columns