Click Element or Link for Submitting Forms or Page Navigation

[doug78645]

I’m having trouble using Click Element or Click Link so I can fill in form fields and then press a Submit button to process the form. This is a very common operation that should be easy to do with Browser Actions, but I’m really struggling to get it to work.

For example, I have a simple Login page with username and password fields as well as a Sign-in button. I believe it’s an Angular web app because there are a lot of ng… elements. My biggest issue if identifying the CSS selector for a page element or label for a link. With Safari, I think I should be able to use Inspect Element for the specific fields, and see the corresponding HTML so I can get the appropriate CSS selector. I understand that a CSS selector for a class has a period prefix (.class), and id has a hash (#id). I have tried entering the username and password (page element) and clicked the Submit link but nothing happens. It would really be helpful to have a simple video tutorial for this scenario and perhaps a nice little example shortcut so we can see how it should work.

Also, I have several pages after the login where I need to click on some list item rows to advance to additional pages, and I’m having trouble doing that too. I’m trying to create some shortcuts that automatically download paperless documents like statements from account websites to make it easier for my wife to do it (if or when I’m not available since she’s not as familiar to all our various websites that I use all the time. Once it’s downloaded, Hazel will automatically rename and move it to my document archive.

Thanks for any guidance or suggestions.

Doug Warren

[czottmann]

I believe it’s an Angular web app because there are a lot of ng… elements

That sounds like an Angular app, alright. It’s possible you’ve run into some issues with the app not accepting “synthetic” clicks, i.e. clicks triggered by code, not an actual input device (mouse, trackpad). Also, if the form is embedded, e.g. in an iframe etc., the clicks might not go through because technically, the iframe would be another document and therefore its contents would be inaccessible by the surrounding page.

If you could give me the link to the page, I could poke around to see what’s going on!

[doug78645]

Here are the browser actions to download a PDF statement from Spectrum (Internet/Voice cable provider):

• Open https://spectrum.net in Safari.
• Get browser reference.
• Home: Click Sign In button.
• Login:
  • Enter Username, Password (Set Form Field with id)
  • Click Sign In button (Click page element)
• Sidebar menu: Select Billing.
• Billing: Click list item link Internet •Home Phone (under Statements).
• Billing Statements: Click link Statement (under Current Statement).
• Bill Details: Click View Printable Statement link.
• Download Viewer: Click Download icon (upper-right).
• Download: Click Save button to save PDF document to Downloads folder.
• More: Sign out
• Close Safari tab.

[czottmann]

That one took me a while. As I’ve said, Angular does a lot of things, and it’s a bit sluggish. In the end it was all about pausing at the right times to give the browser some time to load and initialize the pages. Also, Angular is wrapping the actual form elements like input or button, so for example the form submission button wasn’t #signInBtn but actually #signInBtn button.

Here’s an annotated workflow:

And here’s the usable workflow: Angular app.shortcut (22.7 KB)

It’s a basic demonstration but I hope it’ll help.

Truth be told, automating web sites can be a pain in the ass. A lot of sites are well-built and very WYSIWYG, e.g. elements that look like buttons are actually buttons instead of heavily styled links (a tags). Others are like the Spectrum site. Angular is just one “offender” of many; what looks like a simple page is several layers of styling and wrapping and shadow DOM and Javascript intercepting/ handling/ redirecting user input.

It should be easier but … well.

[doug78645]

Thank you so much for getting that workflow working with Browser Actions! This got me going and I was able to get all the way through the workflow up until the PDF statement download. Also, I had to make a few other tweaks along the way as well.

1. Sometimes, I had trouble where it couldn’t find the Login button. I think this was when I had a previous iteration still in Safari. However, if I remembered to sign-off first so it returned to the Sign In page, then it seemed to work okay.

2. Since my default browser is Arc, I have to be especially careful with these Browser Actions workflows that use Safari or Chrome. I have to make sure that Safari is selected rather than frontmost browser.

3. Also, I have to make sure I select active tab of frontmost window in Safari, rather than specific tab.

4. I assume that should probably Get Browser Reference for Safari up front, rather than explicitly selecting Safari for each action.

5. I’m not sure I understand the Dictionary key type (CSS Selectors) in Set Form Fields (after the Dictionary action with the username and password).

6. I had trouble for a while trying to click on the Billing item in the left sidebar (pseudo-menu). Initially, I was trying to Select Menu, but that failed because it’s not in the app menu bar. Then I was trying to try either Click page element or Click link, but couldn’t get either to work. Eventually, I realized that there’s a Billing link (#manageBillingLink) at the bottom in the Footer, and I use Click page element successfully to get to the Billing Statements page.

7. Also, I had difficulty clicking on the list item in the Statements List and then clicking on the Current Statement on the Billing Detail page.

8. It was tricky clicking Download Viewable Statement (with a download icon) link to view the PDF statement. Eventually, I was able to get it to work if I specifically targeted the download icon (#kite-icon).

9. This opened a new tab in Safari with the PDF document displayed using a URL starting with “blog:https://…”. I’m stuck in the workflow at this point so far, not sure how to automate the final actions to complete the workflow.

10. Manually, I can Select Menu for Export to PDF, which pops up a little Download window. The document name defaults to Unknown, but I can change it to Spectrum-Statement, and select the Downloads folder (if it’s not already selected from the last time), and then pressing the Save button does save it in the Downloads folder (where Hazel renames and moves it to my Documents archive for storage in year-dated subfolders for the Spectrum account.

Any suggestions to get me to the finish line? I want to have download shortcuts for each of my accounts, and I intend to have Download Monthly Statements master shortcut that will cycle through all my paperless accounts and download the PDF statements (or other documents) every month.

I’m primarily doing this for to benefit my wife (and family) after I’m gone since I do all this myself now. I’m trying to make it as easy as possible for them, and any automation would certainly help.

Doug Warren

[czottmann]

@doug78645

• Also, I have to make sure I select active tab of frontmost window in Safari, rather than specific tab.
• I assume that should probably Get Browser Reference for Safari up front, rather than explicitly selecting Safari for each action.

In my example workflow, the first action is basically a combination of those two: An explicit call to the active tab of the frontmost window in Safari. A Get Browser Reference action is unnecessary at this point because it’s explicitly naming Safari.

That specific tab is then explicitly re-used in every follow-up action. Even if the window or the tab goes into the background, this’ll still work because the lookup for “active, frontmost window, Safari” is only done the first time, and from there on the “internal address” of the tab is used, not the “active, frontmost window, Safari” description.

I’m not sure I understand the Dictionary key type (CSS Selectors) in Set Form Fields (after the Dictionary action with the username and password).

Here’s an excerpt of the DOM tree. The custom kite-input element has an ID of “cc-username”, but clicking that doesn’t focus the form field, you have to click the contained input field. And #cc-username input does that: it targets the descendant input field. That’s narrow enough for the browser to find that particular form field; if you just target input then you’d end up with references to all the <input/>s on that page.

I had trouble for a while trying to click on the Billing item in the left sidebar (pseudo-menu). Initially, I was trying to Select Menu, but that failed because it’s not in the app menu bar. Then I was trying to try either Click page element or Click link, but couldn’t get either to work. Eventually, I realized that there’s a Billing link (#manageBillingLink) at the bottom in the Footer, and I use Click page element successfully to get to the Billing Statements page.

It might be possible to just open the billing URL using a Open URL in Browser Tab action there, without having to hunt around the page for it – but only if it’s the same link every time you go there, and the Angular app doesn’t dynamically add session information etc. to it. By that I mean that most sites just check that you’re logged in, without “polluting” the URLs with dynamic session parameters. Some sites do that, and if Spectrum is one of those, then you’ll have to use Click Link to navigate…

This opened a new tab in Safari with the PDF document displayed using a URL starting with “blog:https://…”. I’m stuck in the workflow at this point so far, not sure how to automate the final actions to complete the workflow.

At this point, you could try a Select Menu Item action to trigger the save dialog.

Does that help?

[doug78645]

Thank you. Your explanations are very helpful.

On the Billing Details page, you can view the printable statement at the bottom after the statement summary.

I tried to click on the download icon, which brings up the PDF statement in a new tab.

However, when I select the Export as PDF… menu item to download the statement, I get an error that indicates the app doesn’t have menu bar. But Safari definitely does have a menu bar, so what am I missing.

Here is a screennshot from the web inspector for that download icon.

Thanks.

Doug Warren

[czottmann]

It’s possible that merely the error message is wrong – because there’s no menu item with that title but one with that path (your Select Menu Item action uses title, tho).

Also, and I might be wrong here (I’ve never used that site, mind):

[…] the download icon, which brings up the PDF statement […]

It already is a PDF, so just File > Save As… might be enough? I think exporting a PDF generates an entirely new PDF from the original one, which could lose vital metadata that might be embedded in the PDF.

[doug78645]

You are correct that it should probably be Save As… instead of Export as PDF… since it’s a PDF document in the new tab after clicking on View Printable Statement from the Billing Detail page. I guess I just spotted Export as PDF…, and overlooked the standard Save As… just above (and that’s more appropriate anyway).

Also, Save As… has an assigned keyboard shortcut Cmd-S, so I replaced the Select Menu Item with Simulate Typing S with modifier Cmd.

Earlier, I had forgotten to update the Select Menu Item to path rather than title, when I realized that it needed to be a path File>Save As… rather than just a title Save As… I suppose there could be another menu items under some other main menu option, so it’s probably better to be explicit.

I thought maybe I should insert another action to wait until the PDF document viewer is finished loading before saving it.

Then I would need to add some actions to Click on the Save button in the standard macOS download modal window. Perhaps, I could change Unknown to something like Spectrum-Statement and make sure the Downloads folder is selected. Of course, Hazel will rename the downloaded file based on my naming conventions and move it to the Documents archive in year-dated subfolders by account.

I inserted some Wait to Return actions between each page in the navigation through the website so I could check on the progress in an adjacent window to see where I might be going astray in my webflow.

This was helpful and I discovered that I am able to login to the home page and click on the Billing link, but then I’m failing to click on the statement on the Billing page to go to the Billing Detail page where the View Printable Statement link is available that I want so I can download the PDF statement.

Apparently, I’m not selecting the correct element to make this work.

I have included a screenshot of the Statements card that has a list item for the statement, which only has a single statement listed that I need to click.

This is how I specified the action, but obviously this is NOT working as I thought.

Then the following Billing Statements page has a similar element that needs to be clicked to go to the Billing Detail for the current statement.

Finally, I should get to Billing Detail where I could click on the View Printable Statement to view the PDF statement.

[czottmann]

This is how I specified the action, but obviously this is NOT working as I thought.

Try this selector: billing-statements-card a.kite-list-item__action[tabindex="0"].

It should select the highlighted <a/> tag from your screenshot. Since there are potentially a lot of those links, it specifies to only use those inside the <billing-statements-card/> parent.