API

Top-level package for dask_lsf.

dask_lsf.closesystem(cluster: dask_jobqueue.lsf.LSFCluster, client: distributed.client.Client) → None

Helper function to close cluster and client.

Parameters

  • cluster (LSFCluster) – LSFCluser connected to LSF jobs.

  • client (LSFClient) – LSFClient connected to LSF cluster.

Returns

Return type

None

dask_lsf.get_dashboard_port(client: distributed.client.Client) → int

Returns port on which dask dashboard is running.

Parameters

client (Client) – LSFClient to which LSFCluster it connected.

Returns

Return type

int

dask_lsf.launchlsfjobs(cluster: dask_jobqueue.lsf.LSFCluster, njob: int) → None

Helper function to launch LSF jobs.

Parameters

  • cluster (LSFCluster) – LSFCluster setup for connecting to LSF jobs

  • njobs (int) – Number of jobs to scale upto.

Returns

Return type

None

dask_lsf.setuplsfclient(cluster: dask_jobqueue.lsf.LSFCluster) → distributed.client.Client

Helper function to create LSF client.

Parameters

cluster (LSFCluster) – LSFCluster setup for connecting to LSF jobs

Returns

client

Return type

Client

dask_lsf.setuplsfcluster(queue: str, project_id: str, memory: int, ncores: int, job_extra: Optional[str] = None) → dask_jobqueue.lsf.LSFCluster

Helper function to create LSF Client using DASK module.

Parameters

  • queue (str) – LSF Queue

  • project_id (str) – Project ID. Gets passed to -P argument.

  • memory (int) – Memory per core in GB.

  • ncores (int) – Number of cores per LSF job.

  • job_extra (str) – LSF String Override argument.Default – ‘-R “select[mem >= memory*1000 ] rusage [mem=mem*1000]”’]

Returns

cluster

Return type

LSFCluster

dask_lsf.setupsystem(queue: str, project_id: str, memory: int, ncores: int, njobs: int, wait_for_workers: Optional[int] = None, job_extra: Optional[str] = None) -> (<class 'dask_jobqueue.lsf.LSFCluster'>, <class 'distributed.client.Client'>)

Helper function to setup connection to LSF farm using DASK module.

Parameters

  • queue (str) – LSF Queue

  • project_id (str) – Project ID. Gets passed to -P argument.

  • memory (int) – Memory per core in GB.

  • ncores (int) – Number of cores per LSF job.

  • njobs (int) – Number of LSF jobs.

  • wait_for_workers (int, default None) – Number of LSF jobs to run before executing code. By default the code does not wait for any lsf job.

  • job_extra (str, default '-R "select[mem >= memory*1000 ] rusage [mem=mem*1000]"']) – LSF String Override argument

Returns

  • cluster (LSFCluster)

  • client (Client)

dask_lsf.waitforworkers(client: distributed.client.Client, n: int) → None

Helper function to wait for given number of workers before executing rest of the code.

Parameters

  • client (Client) – LSFClient to which LSFCluster it connected.

  • n (int) – Number of workers to wait before executing rest of the code.

Returns

Return type

None