Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add cn doc for stream.all_reduce #5225

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions docs/api/paddle/distributed/Overview_cn.rst
Original file line number Diff line number Diff line change
Expand Up @@ -87,3 +87,17 @@ paddle.distributed.fleet 是分布式训练的统一入口 API,用于配置分
" :ref:`scatter <cn_api_distributed_scatter>` ", "分发 tensor 到每个进程"
" :ref:`split <cn_api_distributed_split>` ", "切分参数到多个设备"
" :ref:`barrier <cn_api_distributed_barrier>` ", "同步路障,进行阻塞操作,实现组内所有进程的同步"

.. _05:

高级集合通信算法 API
::::::::::::::::::::::

在集群上,对多设备的进程组的参数数据 tensor 进行计算处理。

.. csv-table::
:header: "API 名称", "API 功能"
:widths: 20, 50


" :ref:`all_reduce <cn_api_distributed_stream_all_reduce>` ", "组规约,规约进程组内的 tensor,结果广播至每个进程"
30 changes: 30 additions & 0 deletions docs/api/paddle/distributed/stream/all_reduce_cn.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
.. _cn_api_distributed_stream_all_reduce:

all_reduce
-------------------------------


.. py:function:: paddle.distributed.stream.all_reduce(tensor, op=ReduceOp.SUM, group=None, sync_op=True, use_calc_stream=False)

和 paddle.distributed.all_reduce 的语义相同,进程组内的所有进程在指定 tensor 上所特定的归约操作 (如 sum,reduce 等)。


参数
:::::::::
- tensor (Tensor) - 操作的输入 Tensor,同时也会将归约结果返回至此 Tensor 中。Tensor 的数据类型为:float16、float32、float64、int32、int64。
- op (ReduceOp.SUM|ReduceOp.MAX|ReduceOp.MIN|ReduceOp.PROD,可选) - 归约的具体操作,比如求和,取最大值,取最小值和求乘积,默认为求和归约。
- group (Group,可选) - 在指定通信组中做通信,是可选参数,默认使用全局通信组。
- sync_op (bool, 可选) - 表示做同步或者异步通信,默认进行同步通信。
- use_calc_stream (bool, 可选) - 表示是否在计算流上做通信,默认在通信流上进行通信。

返回
:::::::::
返回一个 task 对象。

警告
:::::::::
该 API 目前只支持在动态图下使用

代码示例
:::::::::
COPY-FROM: paddle.distributed.communication.stream.all_reduce