Kolmogorov–Arnold Networks: Alternative to Multilayer Perceptrons.

{
  "by": "georgehill",
  "descendants": 3,
  "id": 40245696,
  "kids": [
    40245970,
    40245720
  ],
  "score": 2,
  "time": 1714727781,
  "title": "Kolmogorov–Arnold Networks: Alternative to Multilayer Perceptrons.",
  "type": "story",
  "url": "https://arxiv.org/abs/2404.19756"
}

{
  "author": "Ziming Liu",
  "date": "2024-04-30T12:00:00.000Z",
  "description": "Inspired by the Kolmogorov-Arnold representation theorem, we propose Kolmogorov-Arnold Networks (KANs) as promising alternatives to Multi-Layer Perceptrons (MLPs). While MLPs have fixed activation functions on nodes (“neurons”), KANs have learnable activation functions on edges (“weights”). KANs have no linear weights at all -- every weight parameter is replaced by a univariate function parametrized as a spline. We show that this seemingly simple change makes KANs outperform MLPs in terms of accuracy and interpretability. For accuracy, much smaller KANs can achieve comparable or better accuracy than much larger MLPs in data fitting and PDE solving. Theoretically and empirically, KANs possess faster neural scaling laws than MLPs. For interpretability, KANs can be intuitively visualized and can easily interact with human users. Through two examples in mathematics and physics, KANs are shown to be useful collaborators helping scientists (re)discover mathematical and physical laws. In summary, KANs are promising alternatives for MLPs, opening opportunities for further improving today’s deep learning models which rely heavily on MLPs.",
  "image": "https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png",
  "logo": "https://logo.clearbit.com/arxiv.org",
  "publisher": "arXiv.org",
  "title": "KAN: Kolmogorov-Arnold Networks",
  "url": "https://arxiv.org/abs/2404.19756v4"
}

{
  "url": "https://arxiv.org/abs/2404.19756",
  "title": "KAN: Kolmogorov-Arnold Networks",
  "description": "Inspired by the Kolmogorov-Arnold representation theorem, we propose Kolmogorov-Arnold Networks (KANs) as promising alternatives to Multi-Layer Perceptrons (MLPs). While MLPs have fixed activation...",
  "links": [
    "https://arxiv.org/abs/2404.19756v4",
    "https://arxiv.org/abs/2404.19756"
  ],
  "image": "https://static.arxiv.org/icons/twitter/arxiv-logo-twitter-square.png",
  "content": "<div>\n                <p>View a PDF of the paper titled KAN: Kolmogorov-Arnold Networks, by Ziming Liu and 6 other authors</p>\n    <p><a target=\"_blank\" href=\"https://arxiv.org/pdf/2404.19756\">View PDF</a>\n    <a target=\"_blank\" href=\"https://arxiv.org/html/2404.19756v4\">HTML (experimental)</a></p><blockquote>\n            <span>Abstract:</span>Inspired by the Kolmogorov-Arnold representation theorem, we propose Kolmogorov-Arnold Networks (KANs) as promising alternatives to Multi-Layer Perceptrons (MLPs). While MLPs have fixed activation functions on nodes (\"neurons\"), KANs have learnable activation functions on edges (\"weights\"). KANs have no linear weights at all -- every weight parameter is replaced by a univariate function parametrized as a spline. We show that this seemingly simple change makes KANs outperform MLPs in terms of accuracy and interpretability. For accuracy, much smaller KANs can achieve comparable or better accuracy than much larger MLPs in data fitting and PDE solving. Theoretically and empirically, KANs possess faster neural scaling laws than MLPs. For interpretability, KANs can be intuitively visualized and can easily interact with human users. Through two examples in mathematics and physics, KANs are shown to be useful collaborators helping scientists (re)discover mathematical and physical laws. In summary, KANs are promising alternatives for MLPs, opening opportunities for further improving today's deep learning models which rely heavily on MLPs.\n    </blockquote>\n  </div><div>\n      <h2>Submission history</h2><p> From: Ziming Liu [<a target=\"_blank\" href=\"https://arxiv.org/show-email/0c31b93f/2404.19756\">view email</a>]      <br />            <strong><a target=\"_blank\" href=\"https://arxiv.org/abs/2404.19756v1\">[v1]</a></strong>\n        Tue, 30 Apr 2024 17:58:29 UTC (15,986 KB)<br />\n            <strong><a target=\"_blank\" href=\"https://arxiv.org/abs/2404.19756v2\">[v2]</a></strong>\n        Thu, 2 May 2024 16:18:21 UTC (15,986 KB)<br />\n            <strong><a target=\"_blank\" href=\"https://arxiv.org/abs/2404.19756v3\">[v3]</a></strong>\n        Fri, 24 May 2024 22:30:07 UTC (15,991 KB)<br />\n    <strong>[v4]</strong>\n        Sun, 16 Jun 2024 13:34:56 UTC (17,266 KB)<br />\n</p></div>",
  "author": "",
  "favicon": "https://arxiv.org/static/browse/0.3.4/images/icons/favicon-16x16.png",
  "source": "arxiv.org",
  "published": "",
  "ttr": 46,
  "type": "website"
}