unawareness runtime prefetch #572

billie60 · 2024-01-05T09:56:25Z

add prefetchlist store prefetchlist storage service.
Modify the optimizer to publish the access file list as a prefetchlist to the storage service when obtaining it.

imeoer · 2024-01-08T01:29:38Z

tools/prefetchlistStoreService/main.go

@@ -0,0 +1,96 @@
+/*


How about naming it tools/prefetch-distribution?

imeoer · 2024-01-08T01:31:20Z

tools/prefetchlistStoreService/main.go

+func main() {
+	serverCache = Cache{Items: make(map[string]*CacheItem)}
+
+	http.HandleFunc("/api/v1/prefetch/upload", uploadHandler)


Should only accept POST for upload and GET for download, we'd better use the echo web framework.

imeoer · 2024-01-08T01:36:57Z

cmd/optimizer-nri-plugin/main.go

 			Usage:       "whether to overwrite the existed persistent files",
 			Destination: &args.Config.Overwrite,
 		},
+		&cli.StringFlag{


&cli.StringFlag{ Name: "prefetch-distribution-endpoint", Usage: "The service endpoint of prefetch distribution, for example: http://localhost:1323/api/v1/prefetch/upload", Destination: &args.Config.PrefetchDistributionEndpoint, },

Any updates?

imeoer · 2024-01-08T01:39:40Z

cmd/optimizer-nri-plugin/main.go

 	if err != nil {
 		return err
 	}
+	containerName := container.Annotations[containerNameLabel]


Does the annotation exist?

Yes, even without adding annotations in container.yaml, it can still use container.Annotations [containerNameLabel] to get container name

If you only need container name, container.Name is enough.

imeoer · 2024-01-08T01:41:00Z

cmd/optimizer-nri-plugin/main.go

+		}
+	}()
+
 	globalFanotifyServer[imageName] = fanotifyServer


We should protect the map using mutex.

Any updates?

imeoer · 2024-01-08T01:41:33Z

cmd/optimizer-nri-plugin/main.go

+
+	err = postRequest(item, url)
+	if err != nil {
+		return fmt.Errorf("error uploading to server: %w", err)


Use errors.Wrap.

imeoer · 2024-01-08T01:41:36Z

cmd/optimizer-nri-plugin/main.go

+func sendToServer(imageName, prefetchlistPath, containerName, serverURL string) error {
+	data, err := os.ReadFile(prefetchlistPath)
+	if err != nil {
+		return fmt.Errorf("error reading file: %w", err)


Use errors.Wrap.

imeoer · 2024-01-08T01:43:19Z

cmd/optimizer-nri-plugin/main.go

+		return fmt.Errorf("failed to read response body: %w", err)
+	}
+
+	fmt.Println("Server Response:", string(body))


Should check the HTTP status code and return an error if the status code is invalid.

sctb512 · 2024-01-12T02:08:13Z

cmd/optimizer-nri-plugin/main.go

 	if err != nil {
 		return err
 	}
+	containerName := container.Annotations[containerNameLabel]


If you only need container name, container.Name is enough.

sctb512 · 2024-01-12T02:14:34Z

cmd/optimizer-nri-plugin/main.go

+		}
+	}
+
+	item := CacheItem{


Can we define the generic API for different storage service? Sometimes, we may need to send more information to the server.

Also, does the API provide a query endpoint?

sctb512 · 2024-01-12T02:23:10Z

tools/prefetch-distribution/main.go

+		return c.String(http.StatusBadRequest, "Invalid request payload")
+	}
+
+	serverCache.Set(item)


Can we define an interface for difference storage backend?

sctb512 · 2024-01-12T02:29:16Z

cmd/optimizer-nri-plugin/main.go

I guess can we move the fanotify server to an separate component. Because for containerd 1.6.x, the NRI runs in a different construct. If we do this, the NRI for both containerd versions can be simplified to send a request to server.

codecov · 2024-01-22T14:21:07Z

Codecov Report

Attention: Patch coverage is 5.88235% with 16 lines in your changes are missing coverage. Please review.

Project coverage is 34.66%. Comparing base (02b3776) to head (394a502).
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #572      +/-   ##
==========================================
+ Coverage   34.64%   34.66%   +0.01%     
==========================================
  Files          65       65              
  Lines        6552     6552              
==========================================
+ Hits         2270     2271       +1     
+ Misses       3967     3966       -1     
  Partials      315      315

Files	Coverage Δ
pkg/daemon/daemon.go	`0.00% <ø> (ø)`
pkg/system/system.go	`6.46% <ø> (+0.30%)`	⬆️
config/global.go	`19.80% <33.33%> (+0.41%)`	⬆️
pkg/manager/daemon_adaptor.go	`0.00% <0.00%> (ø)`
pkg/manager/manager.go	`0.00% <0.00%> (ø)`
pkg/daemon/config.go	`0.00% <0.00%> (ø)`

imeoer · 2024-02-27T03:14:34Z

.github/workflows/optimizer.yml

+            sudo systemctl restart nydus-snapshotter
+          fi
+          sleep 5
+          NYDUS_VER=v$(curl --header 'authorization: Bearer ${{ secrets.GITHUB_TOKEN }}' -s "https://api.github.com/repos/dragonflyoss/nydus/releases/latest" | jq -r .tag_name | sed 's/^v//')


The --header 'authorization: Bearer ${{ secrets.GITHUB_TOKEN }}' can be removed.

imeoer · 2024-02-27T03:15:37Z

.github/workflows/optimizer.yml

+              echo "Error: --prefetch-files not found in nydus-snapshotter.log"
+              exit 1
+          fi
+          # output=$(sudo cat /etc/nydus/logs/nydus-snapshotter.log | grep "nydusd command")


Should be removed?

imeoer · 2024-02-27T03:16:42Z

cmd/optimizer-nri-plugin/main.go

 			Usage:       "whether to overwrite the existed persistent files",
 			Destination: &args.Config.Overwrite,
 		},
+		&cli.StringFlag{


Any updates?

imeoer · 2024-02-27T03:17:16Z

cmd/optimizer-nri-plugin/main.go

+		}
+	}()
+
 	globalFanotifyServer[imageName] = fanotifyServer


Any updates?

imeoer · 2024-02-27T03:19:21Z

cmd/optimizer-nri-plugin/main.go

+	router *mux.Router
+}
+
+const endpointImageName string = "/api/v1/imagename"


/api/v1/prefetch?reference=$image

imeoer · 2024-02-27T03:20:13Z

cmd/optimizer-nri-plugin/main.go

+		}
+		defer resp.Body.Close()
+
+		responseBody, err := io.ReadAll(resp.Body)


Should check if resp.StatusCode != http.StatusOK { before this.

imeoer · 2024-02-27T03:20:41Z

go.mod

 	k8s.io/utils v0.0.0-20230726121419-3b25d923346b
 )

+require (


Merge these into below.

imeoer · 2024-02-27T03:22:07Z

misc/optimizer/snapshotter-config.toml

@@ -0,0 +1,135 @@
+version = 1


Can we reuse misc/snapshotter/config.toml instead of creating a new one?

1. add prefetchlist store prefetchlist storage service. 2. Modify the optimizer to publish the access file list as a prefetchlist to the storage service when obtaining it. 3. modify http server add lru algo 4. use echo web framework 5. modify based on comments 6. get and post prefetchlist in optimizer

1. send post request to optimizer 2. store prefetchlist 3. add prefetchlist in nydusd

billie60 force-pushed the unawareness-prefetch branch 3 times, most recently from 059d595 to 2994ad5 Compare January 5, 2024 10:40

imeoer reviewed Jan 8, 2024

View reviewed changes

billie60 force-pushed the unawareness-prefetch branch 6 times, most recently from bed33a8 to 01d5e46 Compare January 11, 2024 04:53

sctb512 reviewed Jan 12, 2024

View reviewed changes

billie60 changed the title ~~anawareness runtime prefetch~~ unanawareness runtime prefetch Jan 22, 2024

billie60 force-pushed the unawareness-prefetch branch 4 times, most recently from d0582fa to b96cf48 Compare January 22, 2024 14:14

billie60 force-pushed the unawareness-prefetch branch 13 times, most recently from 4111e29 to dcd0f2f Compare January 29, 2024 12:46

billie60 force-pushed the unawareness-prefetch branch 7 times, most recently from bf97c9a to 1fd5320 Compare January 30, 2024 07:06

billie60 force-pushed the unawareness-prefetch branch 4 times, most recently from 546b4d2 to c2fa245 Compare February 6, 2024 07:58

imeoer changed the title ~~unanawareness runtime prefetch~~ unawareness runtime prefetch Feb 27, 2024

imeoer reviewed Feb 27, 2024

View reviewed changes

billie60 force-pushed the unawareness-prefetch branch 12 times, most recently from dde1a0c to ce605e9 Compare March 4, 2024 10:30

billie60 added 2 commits March 4, 2024 18:28

unawareness prefetch implementation on snapshotter side

34bd531

1. send post request to optimizer 2. store prefetchlist 3. add prefetchlist in nydusd

billie60 force-pushed the unawareness-prefetch branch from ce605e9 to 394a502 Compare March 11, 2024 08:47

modify optimizer test

394a502

imeoer mentioned this pull request Apr 24, 2024

Nydus 2024 Roadmap dragonflyoss/nydus#1575

Closed

unawareness runtime prefetch #572

Are you sure you want to change the base?

unawareness runtime prefetch #572

Uh oh!

Conversation

billie60 commented Jan 5, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Jan 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov bot commented Jan 22, 2024 •

edited

Loading