Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Don't cut bytes from non-address topics + account for empty topics #108

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Don't cut bytes from non-address topics + account for empty topics #108

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 10 additions & 4 deletions internal/storage/clickhouse.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -405,10 +405,16 @@ func addContractAddress(table, query string, contractAddress string) string {
}

func getTopicValueFormat(topic string) string {
toAddressHex := ethereum.HexToAddress(topic)
toAddressPadded := ethereum.LeftPadBytes(toAddressHex.Bytes(), 32)
toAddressTopic := ethereum.BytesToHash(toAddressPadded).Hex()
return toAddressTopic
if topic == "" {
// if there is no indexed topic, indexer stores an empty string
// we shouldn't pad and hexify such an argument then
return ""
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we change this to nil already since we're already working on this or will it blow it up too much?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's a separate thing I wanted to bring up, actually

saw this and thought that maybe we can actually leave it as is, but change the columns to not allow NULLs?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, sounds good. Doesn't really matter '' or null for our use case. We can change the schema and then apply it manually

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since we anyway seem to have no benefit of NULL-ability in these columns and bloom filters are behaving the same way with negation - we can at least optimise some space and performance by completely turning it of

(and, consequently, continuing to use empty strings for topics that don't exist)

}
asBytes := ethereum.FromHex(topic)
// ensure the byte slice is exactly 32 bytes by left-padding with zeros
asPadded := ethereum.LeftPadBytes(asBytes, 32)
result := ethereum.BytesToHash(asPadded).Hex()
return result
}

func (c *ClickHouseConnector) executeAggregateQuery(table string, qf QueryFilter) (map[string]string, error) {
Expand Down
Loading