From 9aa3c7a3e6dae93b8f0d5879ad33bca2563e9863 Mon Sep 17 00:00:00 2001 From: Anthony Wang Date: Wed, 11 Dec 2024 13:07:33 -0500 Subject: Link to TSP repo --- content/posts/solving-shortest-paths-with-transformers.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/content/posts/solving-shortest-paths-with-transformers.md b/content/posts/solving-shortest-paths-with-transformers.md index 32431e7..94457c9 100644 --- a/content/posts/solving-shortest-paths-with-transformers.md +++ b/content/posts/solving-shortest-paths-with-transformers.md @@ -263,6 +263,8 @@ In this post, we've investigated off-distribution generalization behavior of tra We demonstrated mathematically the existence of a transformer computing shortest paths, and also found such a transformer from scratch via gradient descent. We showed that a transformer trained to compute shortest paths between two specific vertices $v_1,v_2$ can be efficiently fine-tuned to compute shortest paths to other vertices that lie on the shortest $v_1$-$v_2$ path, suggesting that our transformers learned representations implicitly carry rich information about the graph. Finally, we showed that the transformer was able to generalize off-distribution quite well in some settings, but less well in other settings. The main conceptual take-away from our work is that it's hard to predict when models will and won't generalize. +You can find our code [here](https://github.com/awestover/transformer-shortest-paths). + ## Appendix ```python -- cgit v1.2.3-70-g09d2