Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SUPPORT]HoodieCleanMetadata and HoodieRestoreMetadata are missing #12023

Open
bithw1 opened this issue Sep 29, 2024 · 1 comment
Open

[SUPPORT]HoodieCleanMetadata and HoodieRestoreMetadata are missing #12023

bithw1 opened this issue Sep 29, 2024 · 1 comment

Comments

@bithw1
Copy link

bithw1 commented Sep 29, 2024

Hi,

I download the Hudi source code from master branch,

Then I run the maven build mvn clean package -DskipTests, it looks that the build succeeds, but the model classes like HoodieCleanMetadata ,HoodieRestoreMetadata are not generated, but some are generated, like HoodieCommitMetadata.

I would ask how to generate these classes, thanks

[INFO] Reactor Summary for Hudi 1.0.0-SNAPSHOT:
[INFO]
[INFO] Hudi ............................................... SUCCESS [ 43.732 s]
[INFO] hudi-tests-common .................................. SUCCESS [ 34.005 s]
[INFO] hudi-io ............................................ SUCCESS [ 27.663 s]
[INFO] hudi-common ........................................ SUCCESS [01:52 min]
[INFO] hudi-hadoop-common ................................. SUCCESS [ 18.850 s]
[INFO] hudi-hadoop-mr ..................................... SUCCESS [ 45.268 s]
[INFO] hudi-sync-common ................................... SUCCESS [ 10.296 s]
[INFO] hudi-hive-sync ..................................... SUCCESS [ 46.931 s]
[INFO] hudi-aws ........................................... SUCCESS [ 28.202 s]
[INFO] hudi-timeline-service .............................. SUCCESS [ 13.416 s]
[INFO] hudi-client ........................................ SUCCESS [  0.153 s]
[INFO] hudi-client-common ................................. SUCCESS [ 47.937 s]
[INFO] hudi-spark-client .................................. SUCCESS [01:31 min]
[INFO] hudi-spark-datasource .............................. SUCCESS [  0.129 s]
[INFO] hudi-spark-common_2.12 ............................. SUCCESS [01:24 min]
[INFO] hudi-spark3-common ................................. SUCCESS [ 32.243 s]
[INFO] hudi-spark3.5.x_2.12 ............................... SUCCESS [01:11 min]
[INFO] hudi-java-client ................................... SUCCESS [ 31.958 s]
[INFO] hudi-spark_2.12 .................................... SUCCESS [01:49 min]
[INFO] hudi-gcp ........................................... SUCCESS [ 19.208 s]
[INFO] hudi-utilities_2.12 ................................ SUCCESS [01:57 min]
[INFO] hudi-utilities-bundle_2.12 ......................... SUCCESS [03:35 min]
[INFO] hudi-cli ........................................... SUCCESS [01:02 min]
[INFO] hudi-flink-client .................................. SUCCESS [ 49.350 s]
[INFO] hudi-datahub-sync .................................. SUCCESS [ 17.559 s]
[INFO] hudi-adb-sync ...................................... SUCCESS [ 13.450 s]
[INFO] hudi-sync .......................................... SUCCESS [  0.123 s]
[INFO] hudi-hadoop-mr-bundle .............................. SUCCESS [ 52.128 s]
[INFO] hudi-datahub-sync-bundle ........................... SUCCESS [ 46.528 s]
[INFO] hudi-hive-sync-bundle .............................. SUCCESS [ 36.086 s]
[INFO] hudi-aws-bundle .................................... SUCCESS [ 44.298 s]
[INFO] hudi-gcp-bundle .................................... SUCCESS [ 34.753 s]
[INFO] hudi-spark3.5-bundle_2.12 .......................... SUCCESS [01:18 min]
[INFO] hudi-presto-bundle ................................. SUCCESS [ 50.886 s]
[INFO] hudi-utilities-slim-bundle_2.12 .................... SUCCESS [01:28 min]
[INFO] hudi-timeline-server-bundle ........................ SUCCESS [01:04 min]
[INFO] hudi-trino-bundle .................................. SUCCESS [ 47.248 s]
[INFO] hudi-examples ...................................... SUCCESS [  0.116 s]
[INFO] hudi-examples-common ............................... SUCCESS [  5.957 s]
[INFO] hudi-examples-spark ................................ SUCCESS [ 39.172 s]
[INFO] hudi-flink-datasource .............................. SUCCESS [  0.138 s]
[INFO] hudi-flink1.19.x ................................... SUCCESS [ 18.550 s]
[INFO] hudi-flink ......................................... SUCCESS [ 45.245 s]
[INFO] hudi-examples-flink ................................ SUCCESS [ 15.046 s]
[INFO] hudi-examples-java ................................. SUCCESS [  8.294 s]
[INFO] hudi-flink1.19-bundle .............................. SUCCESS [01:02 min]
[INFO] hudi-examples-k8s .................................. SUCCESS [02:31 min]
[INFO] hudi-flink1.14.x ................................... SUCCESS [ 28.333 s]
[INFO] hudi-flink1.15.x ................................... SUCCESS [ 20.795 s]
[INFO] hudi-flink1.16.x ................................... SUCCESS [ 17.113 s]
[INFO] hudi-flink1.17.x ................................... SUCCESS [ 21.134 s]
[INFO] hudi-flink1.18.x ................................... SUCCESS [ 31.372 s]
[INFO] hudi-kafka-connect ................................. SUCCESS [ 37.436 s]
[INFO] hudi-kafka-connect-bundle .......................... SUCCESS [02:10 min]
[INFO] hudi-cli-bundle_2.12 ............................... SUCCESS [ 44.114 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  43:04 min
[INFO] Finished at: 2024-09-28T14:46:06+08:00
[INFO] ------------------------------------------------------------------------
@rangareddy
Copy link

Hi @bithw1

I have confirmed that the HoodieCleanMetadata classes are present in both Hudi 0.15.0 and Hudi 1.0.0 jars. Could you please retry with following steps mentioned for 1.0.0 release.

Hudi 0.15.0:

jar tf hudi-common/target/hudi-common-0.15.0.jar | grep HoodieCleanMetadata
org/apache/hudi/avro/model/HoodieCleanMetadata.class
org/apache/hudi/avro/model/HoodieCleanMetadata$Builder.class
org/apache/hudi/avro/model/HoodieCleanMetadata$1.class

Hudi 1.0.0:

git checkout release-1.0.0-beta1
mvn clean install -DskipTests -U 
jar tf hudi-common/target/hudi-common-1.0.0-beta1.jar | grep HoodieCleanMetadata
org/apache/hudi/avro/model/HoodieCleanMetadata.class
org/apache/hudi/avro/model/HoodieCleanMetadata$Builder.class
org/apache/hudi/avro/model/HoodieCleanMetadata$1.class

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: 👤 User Action
Development

No branches or pull requests

3 participants