False positives: "can use", "via ssh"

quackduck commented

2022-04-17 17:29:39 -04:00

(Migrated from github.com)

Of course, I could add these to the false positives list, but maybe there's a better, more general way to tackle these.

TwiN commented

2022-04-17 19:32:59 -04:00

(Migrated from github.com)

Yeah, adding canuse and viassh to the default list of false positives is probably going to be the easiest way to tackle this.

Yeah, adding `canuse` and `viassh` to the default list of false positives is probably going to be the easiest way to tackle this.

quackduck commented

2022-04-21 00:46:07 -04:00

(Migrated from github.com)

True. My issue was more about whether there could be a way to detect these innocent legitimate two word messages.

TwiN commented

2022-04-22 19:47:50 -04:00

(Migrated from github.com)

Yeah there isn't really one besides using the false positives list.

You could create a PR to add them to the default false positives if you'd like: TwiN/go-away@b5570dbc77/falsepositives.go (L4)

Yeah there isn't really one besides using the false positives list. You could create a PR to add them to the default false positives if you'd like: https://github.com/TwiN/go-away/blob/b5570dbc7793ae5f946af94fb431c4a73b64e0b3/falsepositives.go#L4

finnbear commented

2022-05-10 02:59:45 -04:00

(Migrated from github.com)

Yeah there isn't really one besides using the false positives list.

It would take some work on your end, but you could process my comprehensive false positives list in a code generator, as follows:

Read file line by line
Feed each line into goaway
If it detects something, add it as a false positive (or tell me if something bad ended up in the list 😉)

If you're wondering, I generated it using a dictionary search of words and pairs of words, combined with my own additions.

The downside is that my filter operates a bit differently (has some interesting heuristics), and doesn't require certain false positives to be explicitly included in its list. In these cases, you would still need to maintain your own false positive list and/or replicate the dictionary search.

> Yeah there isn't really one besides using the false positives list. It would take some work on your end, but you could process [*my* comprehensive false positives list](https://github.com/finnbear/rustrict/blob/master/src/false_positives.txt) in a code generator, as follows: - Read file line by line - Feed each line into goaway - If it detects something, add it as a false positive (or tell me if something bad ended up in the list :wink:) If you're wondering, I generated it using a [dictionary search of words and pairs of words](https://github.com/finnbear/rustrict/blob/aeb059c3187bffd24e7bb83a3db92bf4eaec9dc5/src/false_positive_finder.rs#L118-L141), combined with [my own additions](https://github.com/finnbear/rustrict/blob/master/src/dictionary_extra.txt). The downside is that my filter operates a bit differently (has some interesting heuristics), and doesn't require certain false positives to be explicitly included in its list. In these cases, you would still need to maintain your own false positive list and/or replicate the dictionary search.

👍 1

quackduck commented

2022-05-10 03:10:17 -04:00

(Migrated from github.com)

Thanks for commenting! @TwiN this could also be a good place to use go:embed (then decode on init() possibly)

(I’m curious: how did you find this thread @finnbear?)

Thanks for commenting! @TwiN this could also be a good place to use go:embed (then decode on init() possibly) (I’m curious: how did you find this thread @finnbear?)

finnbear commented

2022-05-10 03:21:48 -04:00

(Migrated from github.com)

this could also be a good place to use go:embed (then decode on init() possibly)

True! The downside here is that you would be including the entire list, when only a subset is relevant to goaway. A build step/code generator is more work, but could avoid wasting space in the compiled binary by filtering in advance.

(I’m curious: how did you find this thread @finnbear?)

I check in on this repository every once in a while, as it was and is a great source of inspiration for my profanity filters 😃

> this could also be a good place to use go:embed (then decode on init() possibly) True! The downside here is that you would be including the entire list, when only a subset is relevant to goaway. A build step/code generator is more work, but could avoid wasting space in the compiled binary by filtering in advance. > (I’m curious: how did you find this thread @finnbear?) I check in on this repository every once in a while, as it was and is a great source of inspiration for my profanity filters :smiley:

👍 1

quackduck commented

2022-05-10 09:32:35 -04:00

(Migrated from github.com)

True! The downside here is that you would be including the entire list, when only a subset is relevant to goaway.

We could trim the file once as needed

> True! The downside here is that you would be including the entire list, when only a subset is relevant to goaway. We could trim the file once as needed

👍 1

Rows
Columns

False positives: "can use", "via ssh" #28