https://low-hands.github.io/posts/transformer/models/Recent content in Model on Rui's BlogHugo -- gohugo.ioenTue, 27 Jan 2026 16:49:00 -0800https://low-hands.github.io/posts/transformer/models/lstm/Tue, 27 Jan 2026 16:49:00 -0800https://low-hands.github.io/posts/transformer/models/lstm/<p>pending</p>https://low-hands.github.io/posts/transformer/models/rnn/Mon, 26 Jan 2026 23:57:00 -0800https://low-hands.github.io/posts/transformer/models/rnn/<h1 id="结构">结构</h1> <p><img src="https://low-hands.github.io/posts/transformer/models/rnn/structure.png" alt="structure"> 纵向看，每一列相当于一个小的传统神经网络</p> <p>x：输入层，长度为word embedding的维度</p> <p>U：输入层到隐藏层的权重</p> <p>s：隐藏层 = f(U * xt + W * s(t-1) + bias),即输入层的计算结果加上上一时刻的结果乘上一个权重W</p> <p>V: 隐藏层到输出层的权重</p> <p>o：输出 = softmax(V * st + bias)</p> <h1 id="注意">注意</h1> <ul> <li>由公式可见，每个隐藏层s的值不仅依赖于当前时刻的输入，而且依赖于<strong>上一个时刻的的结果s(t-1)</strong></li> <li>U,V,W都是共享的</li> <li>不一定每一个时刻都要有一个输出，也就是说不是每一个小神经网络都必须有输出，它可以是不完整的</li> </ul> <h1 id="举例">举例</h1> <p><img src="https://low-hands.github.io/posts/transformer/models/rnn/8397aaf6-1669-46b3-8465-791ed748404f.png" alt="example"></p> <p>这是一个时间序列做预测的例子，一共有三天的数据，预测明天的数据</p> <p>w1 = U = 1.8， w2 = W = -0.5，w3 = V = 1.1</p> <p>x0 = 1, x1 = 0.5, x2 = 0.5 这里的输入只是数字而不是向量</p> <h2 id="day0">Day0</h2> <p>s0 = f(1.8 * 1 + -0.5 * 0) = f(1.8) = 1.8</p>