Logo Rui's Blog
  • Home
  • About
  • Skills
  • Experiences
  • Education
  • More
    Featured Posts Recent Posts
  • Posts
  • Notes
  • English
    English 简体中文
  • Dark Theme
    Light Theme Dark Theme System Theme
Logo Inverted Logo
  • Posts
  • Markdown Sample
  • Shortcodes Sample
  • Internship
    • MiniMind
  • LLM
    • LLM architecture
      • Encoder-only
  • Paper Reading
    • dLLM
    • YAN
  • Pytorch
  • Pytorch
  • Transformer
    • Tokenizer
    • Word Embedding
    • Model
      • LSTM
      • RNN
  • algorithm
    • Array
      • Binary Search
    • Backtracking
    • Binary Tree
    • Dyanmic Programming
      • Multi-dimensional Dynamic Programming
    • Graph Theory
    • Linked List
    • Two-pointers
      • Leetcode 27. Remove Element
      • Leetcode 541. Reverse String II
      • Leetcode 344. Reverse String
      • Sliding Window
Hero Image
LSTM

pending

    Tuesday, January 27, 2026 | 1 minute Read
    Hero Image
    RNN

    结构 纵向看,每一列相当于一个小的传统神经网络 x:输入层,长度为word embedding的维度 U:输入层到隐藏层的权重 s:隐藏层 = f(U * xt + W * s(t-1) + bias),即输入层的计算结果加上上一时刻的结果乘上一个权重W V: 隐藏层到输出层的权重 o:输出 = softmax(V * st + bias) 注意 由公式可见,每个隐藏层s的值不仅依赖于当前时刻的输入,而且依赖于上一个时刻的的结果s(t-1) U,V,W都是共享的 不一定每一个时刻都要有一个输出,也就是说不是每一个小神经网络都必须有输出,它可以是不完整的 举例 这是一个时间序列做预测的例子,一共有三天的数据,预测明天的数据 w1 = U = 1.8, w2 = W = -0.5,w3 = V = 1.1 x0 = 1, x1 = 0.5, x2 = 0.5 这里的输入只是数字而不是向量 Day0 s0 = f(1.8 * 1 + -0.5 * 0) = f(1.8) = 1.8

      Monday, January 26, 2026 | 1 minute Read
      Navigation
      • About
      • Skills
      • Experiences
      • Education
      • Featured Posts
      • Recent Posts
      Contact me:
      • ruig0401@gmail.com
      • low-hands
      • Rui
      • +1 5875886800

      Liability Notice: This theme is under MIT license. So, you can use it for non-commercial, commercial, or private uses. You can modify or distribute the theme without requiring any permission from the theme author. However, the theme author does not provide any warranty or takes any liability for any issue with the theme.


      Toha Theme Logo Toha
      © 2026 Copyright.
      Powered by Hugo Logo