• Signed and dated written informed consent

• Subjects \>= 18 years of age

• Eastern Cooperative Oncology Group (ECOG) performance status 0 or 1

• Clinical stage IV invasive breast cancer or unresectable locoregional recurrence of invasive breast cancer meeting the following criteria:

‣ Estrogen receptor (ER)/progesterone receptor (PR)-negative (=\< 5% cells) by immunohistochemistry (IHC) and human epidermal grow (HER2) negative (by IHC or fluorescence in situ hybridization (FISH))

⁃ Measurable disease as defined by Response Evaluation Criteria in Solid Tumors (RECIST) version 1.1 criteria and which can be followed by computed tomography (CT) or magnetic resonance imaging (MRI). A measurable lytic bone lesion(s) and/or skin lesion(s) are allowed. Skin lesions must also be followed by photography with measuring tools within the photograph at each tumor evaluation time point. Ultrasound may be used to follow breast lesions not visible by CT following discussion with Study Chair

⁃ Amenable to biopsy at the time of study entry

⁃ Known tumor/immune cell PD-L1 status by any assay

• Adequate organ function including:

‣ Cardiac ejection fraction at or above the institutional lower limit of normal, as assessed by either echocardiogram or multigated acquisition (MUGA) scan

⁃ Absolute neutrophil count (ANC) \>= 1.0 x 10\^9/L (may have received growth factor)

⁃ Platelets \>= 100 x 10\^9/L

⁃ Hemoglobin \>= 9 g/dL (may have been transfused)

⁃ Total serum bilirubin =\< 1.5 times upper limit of normal (ULN)

⁃ Aspartate aminotransferase (AST/serum glutamic oxaloacetic transaminase (SGOT)) and alanine aminotransferase (ALT/serum glutamate pyruvate transaminase (SGPT)) =\< 2.5 x ULN (or =\< 5 x ULN if liver metastases are present)

⁃ Serum creatinine =\< 1.5 x ULN or estimated creatinine clearance \>= 50 mL/min as calculated using the Cockcroft-Gault (CG) equation

⁃ Prothrombin time (PT)/international normalized ratio (INR) =\< 1.5 x ULN

⁃ Amylase =\< 1 x ULN testing is only required in patients with a history of pancreatic disorders (Abnormality not of pancreatic origin is allowed)

⁃ Participants with treated and controlled hypo or hyperthyroidism are eligible.

• Male and female patients of childbearing potential must agree to use at least two methods of acceptable contraception from 15 days prior to first trial treatment administration until at least 30 days after study participant's final dose of study drug(s)

• \* NOTE: Females of childbearing potential are defined as those who are not surgically sterile or post-menopausal (i.e., patient has not had a bilateral tubal ligation, a bilateral oophorectomy, or a complete hysterectomy; or has not been amenorrhoeic for 12 months without an alternative medical cause). Post-menopausal status in females under 55 years of age should be confirmed with a serum follicle-stimulating hormone (FSH) level within laboratory reference range for postmenopausal women

• Patients unable to read/write in English are eligible to participate in the overall study but will not participate in the Patient-Reported Outcome questionnaires throughout the trial.

• Re-enrollment of a subject that has discontinued the study as a pre-randomization screen failure (i.e., a consented patient who was not randomized and did not receive any study treatment) is permitted. If re-enrolled, the subject must be re-consented. Only the screening procedures performed outside of protocol-specified timing must be repeated; if biopsies and correlative blood samples were already obtained, and patient has not received any systemic anti-cancer therapy since they were obtained, they do not need to be repeated.