[WP Smush Pro] Smush Bug: smush attempting to operate on local file paths when s3 is active…bug

I have recently upgraded to to php 7.2 and have been systematically searching for issues…here’s the latest, and this is important.

I am very concerned about the following function:

function get_attached_file( $attachment_id ) {
if ( empty( $attachment_id ) ) {
return false;
}

$file_path = get_attached_file( $attachment_id );
if ( ! empty( $file_path ) && strpos( $file_path, 's3' ) !== false ) {
$file_path = get_attached_file( $attachment_id, true );
}

return $file_path;
}

I have S3 active…WITH cname dns set up to pass through the buck

files.networkname.com -> files.networkname.com.s3-us-west-1.amazonaws.com

meaning my s3 paths will reference files.mynetworkname.com instead of a really long ugly name with s3 in it. Its the right way to set things up and can be easily accomplished and should be considered a standard use case.

Your function appears to be looking for s3 in the file path…to make a conclusion of weather or not to get the full server path of where the image would be….this is really bad and can lead to other issues in your code because this function will return the local path of an image that does not exist, not something that it looks like you are expecting.

For example, in the file:

lib/class-wp-smush-resize.php

line 105:

$file_path = $wpsmush_helper->get_attached_file( $id );

even when S3 plugin is active in the settings, $file_path is set to a local file path…

which then the causes this line $this->is_animated( $file_path ) to run the function is_animated to run, and on line 413

$filecontents = file_get_contents( $file_path );

which will fail because there will NEVER be a local file to check….

BUG BUG BUG BUG that needs to be immediately reviewed.

2ndly…this was running when I went to https://networkname.com/subsite/wp-admin/upload.php

which loads in very slow and i’m seeing all the error messages in my logs….why is is_animated even running? And what other overhead is running unnecessarily because of smush here.

Please review the code lines i have given you. Please check to make sure of the following things:

1. Smush is not running unnecessarily on all page loads. I have revealed both domain mapping plugin and ultimate branding are very guilty of that….this appears to be guilty of that too and clogging up servers unnecessarily.

2. in the file lib/class-wp-smush-helper.php line 32:

function get_attached_file in the class WpSmushHelper is straight up broken and using failing logic and needs to be fix it asap as well as a review what other functions are reliant on it.

  • Ben
    • The Incredible Smush

    I can also confirm the error messaage in logs only happenw hen image resize is toggled to “on”

    So…image resize is running some things on every page load when its not supposed to be running…super dangerous. Should only be running resize stuff when bulk smush is pressed or a single smush or an item is uploaded, right?

    You need to catch that issue AND the s3 issue of wrong paths.

  • Adam
    • Support Gorilla

    Hello Ben

    I hope you’re having a nice day!

    and i’m seeing all the error messages in my logs (…:wink: I can also confirm the error messaage in logs only happenw hen image resize is toggled to “on”

    Would you please share the error message(s) or, better yet, the error log with us, please? I already talked to plugin’s lead developer and he’ll be testing it but he needs that information as well.

    Kind regards,

    Adam

  • Ben
    • The Incredible Smush

    here is a small excerpt:

    [23-Sep-2018 22:04:28 UTC] PHP Warning:  file_get_contents(/srv/users/runserver/apps/wordpress/public/wp-content/uploads/sites/4/2017/11/received_10214664690597220.png): failed to open stream: No such file or directory in /srv/users/runserver/apps/wordpress/public/wp-content/plugins/wp-smush-pro/lib/class-wp-smush-resize.php on line 413
    [23-Sep-2018 22:04:28 UTC] PHP Warning: file_get_contents(/srv/users/runserver/apps/wordpress/public/wp-content/uploads/sites/4/2017/11/20171122_151444-1.jpg): failed to open stream: No such file or directory in /srv/users/runserver/apps/wordpress/public/wp-content/plugins/wp-smush-pro/lib/class-wp-smush-resize.php on line 413
    [23-Sep-2018 22:04:28 UTC] PHP Warning: file_get_contents(/srv/users/runserver/apps/wordpress/public/wp-content/uploads/sites/4/2017/11/20171122_151444-1.jpg): failed to open stream: No such file or directory in /srv/users/runserver/apps/wordpress/public/wp-content/plugins/wp-smush-pro/lib/class-wp-smush-resize.php on line 413
    [23-Sep-2018 22:04:28 UTC] PHP Warning: file_get_contents(/srv/users/runserver/apps/wordpress/public/wp-content/uploads/sites/4/2017/11/20171122_152641-1.jpg): failed to open stream: No such file or directory in /srv/users/runserver/apps/wordpress/public/wp-content/plugins/wp-smush-pro/lib/class-wp-smush-resize.php on line 413
    [23-Sep-2018 22:04:28 UTC] PHP Warning: file_get_contents(/srv/users/runserver/apps/wordpress/public/wp-content/uploads/sites/4/2017/11/20171122_152641-1.jpg): failed to open stream: No such file or directory in /srv/users/runserver/apps/wordpress/public/wp-content/plugins/wp-smush-pro/lib/class-wp-smush-resize.php on line 413
    [23-Sep-2018 22:04:28 UTC] PHP Warning: file_get_contents(/srv/users/runserver/apps/wordpress/public/wp-content/uploads/sites/4/2017/11/20171122_152908-1.jpg): failed to open stream: No such file or directory in /srv/users/runserver/apps/wordpress/public/wp-content/plugins/wp-smush-pro/lib/class-wp-smush-resize.php on line 413
    [23-Sep-2018 22:04:28 UTC] PHP Warning: file_get_contents(/srv/users/runserver/apps/wordpress/public/wp-content/uploads/sites/4/2017/11/20171122_152908-1.jpg): failed to open stream: No such file or directory in /srv/users/runserver/apps/wordpress/public/wp-content/plugins/wp-smush-pro/lib/class-wp-smush-resize.php on line 413

  • Ben
    • The Incredible Smush

    Also you can also run a a print to log in that block of code and trace back the scenarios it

    and for the other issue of testing for s3, the line of code:

    strpos( $file_path, 's3' ) !== false

    I believe is meant to test if the file path is on an s3 server, however it is not a proper test anymore and I believe it could lead to false results.

  • Ben
    • The Incredible Smush

    Umesh Kumar Adam Czajczyk

    I’ve reviewed more and I now see whats going on…

    When in the media view grid view it can show a lot of images. It looks like you are attempting to add a smush or re-smush button on load for all the images, which requires qualifing if an image should be resmush via if it is animated and possibly other factors. If the resize feature is made avail, its going to attempt to check if the image is an imated image and if that image is on s3 it may have to download that huge image to the server in memory to test if its animated. There might be some clients that upload their samsung s9 5mb-7mb photo files directly to their wordpress site and as a result that looks to be a scenario that can add heavy load. The first 8rows x 8columns x 5mb …that big to test before a page display

    Anyways possibly should be a way to qualify if the smush button should show when viewing all the images in the media gallery. One image at a time is not a problem but a bunch of images at once is causing serious problems.

    So for the function show_resmush, i think getting rid of this on line 1999 in file lib/class-wp-smush.php

    $wpsmush_resize->should_resize( $id )

    Will solve my immediate issue till the code is fixed.

    Thoughts? And are there any other trouble areas you can think of?

  • Adam
    • Support Gorilla

    Hello Ben

    Thank you for providing requested log and additional information.

    I passed that to the lead developer of Smush Pro, he read your posts and decided to consider it as a bug so it’s already on a bug list and he and his team will look into it. He’ll need to run some more tests and look for a steady solution so please note that this might take some time but I just spoke to him directly and he confirmed they need to and will address the issue.

    Kind regards,

    Adam

  • Ben
    • The Incredible Smush

    okay hope it works out.

    Another note I’ve brought up before…i would really like on the hub to have a list of “known issues” for the plugins I have loaded.

    If wpmudev has an internal bug list…i would be fair to paying members to have access to that list so that we aren’t surprised by bugs. It will save both wpmudev tech support and…most importantly…users of your plugs. This transparency would be beneficial to all parties. +1

  • Adam
    • Support Gorilla

    Hello Ben

    Another note I’ve brought up before…i would really like on the hub to have a list of “known issues” for the plugins I have loaded. (…:wink:

    May I ask you to put this into a separate ticket of its own, preferably in our “Features and Feedback” forum? We’ll then be able to discuss that separately, let other members follow up and share their opinions and, in general, keep it better visible while focusing on the main query in this thread. I’d appreciate this a lot!

    Thanks in advance :slight_smile:

    Best regards,

    Adam

  • Ben
    • The Incredible Smush

    Looks like a lot of work was done, but still confused…I’m still seeing:

    if ( ! empty( $file_path ) && strpos( $file_path, 's3' ) !== false )

    is that line going work as expected if say a file has the “s3” in the name or if the url for s3 bucket doesn’t have s3 in it?

    Thanks!

  • Ben
    • The Incredible Smush

    Umesh Kumar Adam Czajczyk

    Just did the update to the latest plugin on a s3 website network.

    Viewing the media gallery…crashes things. Error logs show the following:

    [27-Sep-2018 16:20:11 UTC] PHP Fatal error:  Uncaught Error: Call to undefined method Amazon_S3_And_CloudFront::is_attachment_served_by_provider() in /srv/users/serverpilot/apps/wordpress/public/wp-content/plugins/wp-smush-pro/lib/integrations/class-wp-smush-s3.php:265
    Stack trace:
    #0 /srv/users/serverpilot/apps/wordpress/public/wp-content/plugins/wp-smush-pro/lib/integrations/class-wp-smush-s3.php(438): WpSmushS3->is_image_on_s3(2278)
    #1 /srv/users/serverpilot/apps/wordpress/public/wp-includes/class-wp-hook.php(286): WpSmushS3->backup_exists_on_s3(false, 2278, 's3uswest1://fil...')
    #2 /srv/users/serverpilot/apps/wordpress/public/wp-includes/plugin.php(203): WP_Hook->apply_filters(false, Array)
    #3 /srv/users/serverpilot/apps/wordpress/public/wp-content/plugins/wp-smush-pro/lib/class-wp-smush.php(1754): apply_filters('smush_backup_ex...', false, 2278, 's3uswest1://fil...')
    #4 /srv/users/serverpilot/apps/wordpress/public/wp-content/plugins/wp-smush-pro/lib/class-wp-smush.php(1125): WP_Smush->show_restore_option(2278, Array)
    #5 /srv/users/serverpilot/apps/ in /srv/users/serverpilot/apps/wordpress/public/wp-content/plugins/wp-smush-pro/lib/integrations/class-wp-smush-s3.php on line 265

    Using version: 1.4.3 http://wordpress.org/extend/plugins/amazon-s3-and-cloudfront/

    Latest version 2.0 released yesterday I am still testing. But this plugin needs to maintain compatibility with a version that was live 2 days ago….but i am guessing you have another issue.

    please tell the developer immediately, thanks!

  • Adam
    • Support Gorilla

    Hello Ben!

    If it comes to your original report, there’s been no Smush release yet addressing it. That’s “on the table” now and I see it’s being worked on, however no code for that has been released yet. The very last release of Smush was basically a version reverting one before last as it caused some unexpected issue on some setups.

    So, that “reverted” release could have, indeed, include some code that might affected S3 even though it wasn’t containing a direct fix for the issue reported by you. I’m not sure if what I’m saying is clear but I believe you get what I’m trying to say :slight_smile: Basically, these S3 issues are in development and we’ll be address in upcoming releases (though I can’t guarantee the nearest one).

    As for the S3 plugin. We usually try to maintain compatibility with various plugins and themes (not to mention WP core) as much “back” as possible but, well, it’s not always possible. We can only guarantee compatibility with the most recent one if it comes to 3rd party plugins as we don’t have any “impact” on their code and sometimes that code changes too much. I’m not sure if this is the case here so don’t take my words as an “ultimate statement” here, please, as ultimately it’s all about what our developers will do. They’re working on that so we need to wait for the release containing a fix, which, I hope, will happen soon.

    Kind regards,

    Adam

  • Ben
    • The Incredible Smush

    Adam Czajczyk

    Let me help clarify a few things:

    in the new 2.0 version of amazon s3 plugin:

    get_provider_client was changed to get_provider_client

    AND

    is_attachment_served_by_s3 was changed to is_attachment_served_by_provider

    You’re last smush update, updated those methods to the new 2.0 version. You can check the wordpress.org stats…maybe 6.3% of the world updated their plugin…which is why the update broke things for most users, because most users aren’t using version 2.0. (Because the s3 pluginis big update and may have some kinks, people are waiting for version 2.1 to let the kinks be worked out.)

    So an example of usage in: lib/integrations/class-wp-smush-s3.php

    $s3_object = $as3cf->is_attachment_served_by_s3( $attachment_id, true );

    should be re-rewritten like this:

    if ( method_exists( $as3cf, 'is_attachment_served_by_s3' ) ) {
    $s3_object = $as3cf->is_attachment_served_by_s3( $attachment_id, true );
    } else {
    $s3_object = $as3cf->is_attachment_served_by_provider( $attachment_id, true );
    }

    But…he didn’t. Pretty basic and that looks to be the fail.

    Now fixing this in what I can see about 6 places takes maybe 15 min…so not sure why the delay in fixing it. I told the support tech exactly what to do and what to tell the developer.

    So a little frustrating not just on the mistake, but also on follow through. Trying to help you guys and the community.

  • Ben
    • The Incredible Smush

    here are my fixes in the file: lib/integrations/class-wp-smush-s3.php

    <?php
    /**
    * S3 integration: WpSmushS3 class
    *
    * @package WP_Smush
    * @subpackage S3
    * @version 2.7
    *
    * @author Umesh Kumar <umesh@incsub.com>
    *
    * @copyright (c) 2017, Incsub (http://incsub.com)
    */

    if ( ! class_exists( 'WpSmushS3' ) ) {

    /**
    * Class WpSmushS3
    */
    class WpSmushS3 {

    /**
    * Module slug.
    *
    * @since 2.8.1
    *
    * @var string $module
    */
    private $module = 's3';

    /**
    * WpSmushS3 constructor.
    */
    function __construct() {
    add_action( 'admin_init', array( $this, 'init' ), 5 );

    // Hook at the end of setting row to output a error div.
    add_action( 'smush_setting_column_right_inside', array( $this, 's3_setup_message' ) );
    }

    /**
    * Init actions
    */
    function init() {
    global $wp_smush;

    // Filters the setting variable to add S3 setting title and description.
    add_filter( 'wp_smush_settings', array( $this, 'register' ), 6 );

    // Filters the setting variable to add S3 setting in premium features.
    add_filter( 'wp_smush_integration_settings', array( $this, 'add_setting' ), 1 );

    // Disable setting.
    add_filter( 'wp_smush_integration_status_' . $this->module, array( $this, 'setting_status' ) );

    // Return if not a pro user or the S3 plugin is not installed.
    if ( ! $wp_smush->validate_install() || ( ! class_exists( 'Amazon_S3_And_CloudFront' ) && ! class_exists( 'Amazon_S3_And_CloudFront_Pro' ) ) ) {
    return;
    }

    // Check if the file exists for the given path and download.
    add_action( 'smush_file_exists', array( $this, 'maybe_download_file' ), 10, 3 );

    // Check if the backup file exists.
    add_filter( 'smush_backup_exists', array( $this, 'backup_exists_on_s3' ), 10, 3 );

    add_action( 'smush_setting_column_right_inside', array( $this, 'additional_notice' ) );

    // Show submit button when a pro user and the S3 plugin is installed.
    add_filter( 'wp_smush_integration_show_submit', '__return_true' );
    }

    /**
    * Filters the setting variable to add S3 setting title and description.
    *
    * @param array $settings Settings array.
    *
    * @return mixed
    */
    function register( $settings ) {
    $plugin_url = esc_url( 'https://wordpress.org/plugins/amazon-s3-and-cloudfront/' );
    $settings[ $this->module ] = array(
    'label' => esc_html__( 'Enable Amazon S3 support', 'wp-smushit' ),
    'short_label' => esc_html__( 'Amazon S3', 'wp-smushit' ),
    'desc' => sprintf(
    esc_html__( "Storing your image on S3 buckets using %sWP Offload S3%s? Smush can detect and smush those assets for you, including when you're removing files from your host server.", 'wp-smushit' ),
    "<a href='" . $plugin_url . "' target = '_blank'>",
    '</a>',
    '<b>',
    '</b>'
    ),
    );

    return $settings;
    }

    /**
    * Append S3 in PRO feature list
    *
    * @param array $pro_settings Pro settings.
    *
    * @return array
    */
    function add_setting( $pro_settings ) {
    if ( ! isset( $pro_settings[ $this->module ] ) ) {
    $pro_settings[] = $this->module;
    }

    return $pro_settings;
    }

    /**
    * Prints the message for S3 setup
    *
    * @param string $setting_key Settings key.
    *
    * @return null
    */
    function s3_setup_message( $setting_key ) {
    // Return if not S3.
    if ( $this->module !== $setting_key ) {
    return;
    }

    global $as3cf, $wp_smush, $wpsmush_settings;

    $is_pro = $wp_smush->validate_install();

    // If S3 integration is not enabled, return.
    $setting_val = $is_pro ? $wpsmush_settings->settings[ $this->module ] : 0;

    // If integration is disabled when S3 offload is active, do not continue.
    if ( ! $setting_val && is_object( $as3cf ) ) {
    return;
    }

    $class = $message = '';

    // If S3 offlocad global variable is not available, plugin is not active.
    if ( ! is_object( $as3cf ) ) {
    $message = __( 'To use this feature you need to install WP Offload S3 and have an Amazon S3 account setup.', 'wp-smushit' );
    } elseif ( ! method_exists( $as3cf, 'is_plugin_setup' ) ) {
    // Check if in case for some reason, we couldn't find the required function.
    $class = ' sui-notice-warning';
    $support_url = esc_url( 'https://wpmudev.com/contact' );
    $message = sprintf( esc_html__( 'We are having trouble interacting with WP Offload S3, make sure the plugin is activated. Or you can %sreport a bug%s.', 'wp-smushit' ), '<a href="' . $support_url . '" target="_blank">', '</a>' );
    } elseif ( ! $as3cf->is_plugin_setup() ) {
    // Plugin is not setup, or some information is missing.
    $class = ' sui-notice-warning';
    $configure_url = $as3cf->get_plugin_page_url();
    $message = sprintf( esc_html__( 'It seems you haven’t finished setting up WP Offload S3 yet. %sConfigure it now%s to enable Amazon S3 support.', 'wp-smushit' ), '<a href="' . $configure_url . '" target="_blank">', '</a>' );
    } else {
    // S3 support is active.
    $class = ' sui-notice-info';
    $message = __( 'Amazon S3 support is active.', 'wp-smushit' );
    }

    // Return early if we don't need to do anything.
    if ( empty( $message ) || ! $is_pro ) {
    return;
    }

    echo '<div class="sui-notice' . $class . ' smush-notice-sm"><p>' . $message . '</p></div>';
    }

    /**
    * Show additional notice if the required plugins are not istalled.
    *
    * @since 2.8.0
    *
    * @param string $name Setting name.
    */
    public static function additional_notice( $name ) {
    // If we don't have free or pro version for WP Offload S3, return.
    if ( 's3' === $name && ! class_exists( 'Amazon_S3_And_CloudFront' ) && ! class_exists( 'Amazon_S3_And_CloudFront_Pro' ) ) { ?>
    <div class="sui-notice sui-notice-sm">
    <p>
    <?php esc_html_e( 'To use this feature you need to install WP Offload S3 and have an Amazon S3 account setup.', 'wp-smushit' ); ?>
    </p>
    </div>
    <?php
    }
    }

    /**
    * Error message to show when S3 support is required.
    *
    * Show a error message to admins, if they need to enable S3 support. If "remove files from
    * server" option is enabled in WP Offload S3 plugin, we need WP Smush Pro to enable S3 support.
    *
    * @return mixed
    */
    function s3_support_required_notice() {

    global $wpsmushit_admin, $wpsmush_settings;

    // Do not display it for other users.
    // Do not display on network screens, if networkwide option is disabled.
    if ( ! current_user_can( 'manage_options' ) || ( is_network_admin() && ! $wpsmush_settings->settings['networkwide'] ) ) {
    return true;
    }

    // Do not display the notice on Bulk Smush Screen.
    global $current_screen;
    if ( ! empty( $current_screen->base ) && 'toplevel_page_smush' != $current_screen->base && 'gallery_page_wp-smush-nextgen-bulk' != $current_screen->base && 'toplevel_page_smush-network' != $current_screen->base ) {
    return true;
    }

    // If already dismissed, do not show.
    if ( 1 == get_site_option( 'wp-smush-hide_s3support_alert' ) ) {
    return true;
    }

    // Return early, if support is not required.
    if ( ! $this->s3_support_required() ) {
    return true;
    }

    // Settings link.
    $settings_link = is_multisite() && is_network_admin() ? network_admin_url( 'admin.php?page=smush' ) : menu_page_url( 'smush', false );

    if ( $wpsmushit_admin->validate_install() ) {
    // If premium user, but S3 support is not enabled.
    $message = sprintf( __( "We can see you have WP Offload S3 installed with the <strong>Remove Files From Server</strong> option activated. If you want to optimize your S3 images you'll need to enable the <a href='%s'><strong>Amazon S3 Support</strong></a> feature in Smush's settings.", 'wp-smushit' ), $settings_link );
    } else {
    // If not a premium user.
    $message = sprintf( __( "We can see you have WP Offload S3 installed with the <strong>Remove Files From Server</strong> option activated. If you want to optimize your S3 images you'll need to <a href='%s'><strong>upgrade to Smush Pro</strong></a>", 'wp-smushit' ), esc_url( 'https://wpmudev.com/project/wp-smush-pro' ) );
    }

    echo '<div class="sui-notice sui-notice-warning wp-smush-s3support-alert"><p>' . $message . '</p><span class="sui-notice-dismiss"><a href="#">' . esc_html__( 'Dismiss', 'wp-smushit' ) . '</a></span></div>';
    }

    /**
    * Check if S3 support is required for Smush.
    *
    * @return bool
    */
    function s3_support_required() {

    global $wpsmush_settings, $wpsmushit_admin, $as3cf;

    // Check if S3 offload plugin is active and delete file from server option is enabled.
    if ( ! is_object( $as3cf ) || ! method_exists( $as3cf, 'get_setting' ) || ! $as3cf->get_setting( 'remove-local-file' ) ) {
    return false;
    }

    // If not Pro user or S3 support is disabled.
    return ( ! $wpsmushit_admin->validate_install() || ! $wpsmush_settings->settings[ $this->module ] );
    }

    /**
    * Checks if the given attachment is on S3 or not, Returns S3 URL or WP Error
    *
    * @param $attachment_id
    *
    * @return bool|false|string
    *
    */
    function is_image_on_s3( $attachment_id = '' ) {
    global $as3cf;
    if ( empty( $attachment_id ) || ! is_object( $as3cf ) ) {
    return false;
    }

    //If we only have the attachment id
    // ben ben $full_url = $as3cf->is_attachment_served_by_provider( $attachment_id, true );
    if ( method_exists( $as3cf, 'is_attachment_served_by_s3' ) ) {
    $full_url = $as3cf->is_attachment_served_by_s3( $attachment_id, true );
    } else {
    $full_url = $as3cf->is_attachment_served_by_provider( $attachment_id, true );
    }
    //If the filepath contains S3, get the s3 URL for the file
    if ( ! empty( $full_url ) ) {
    $full_url = $as3cf->get_attachment_url( $attachment_id );
    } else {
    $full_url = false;
    }

    return $full_url;

    }

    /**
    * Download a specified file to local server with respect to provided attachment id
    * and/or Attachment path
    *
    * @param $attachment_id
    *
    * @param array $size_details
    *
    * @param string $uf_file_path
    *
    * @return string|bool Returns file path or false
    *
    */
    function download_file( $attachment_id, $size_details = array(), $uf_file_path = '' ) {
    global $wp_smush, $wpsmush_settings;
    if ( empty( $attachment_id ) || ! $wpsmush_settings->settings[ $this->module ] || ! $wp_smush->validate_install() ) {
    return false;
    }

    global $as3cf;
    $renamed = $s3_object = $s3_url = $file = false;

    //If file path wasn't specified in argument
    $uf_file_path = empty( $uf_file_path ) ? get_attached_file( $attachment_id, true ) : $uf_file_path;

    //If we have plugin method available, us that otherwise check it ourselves
    if ( method_exists( $as3cf, 'is_attachment_served_by_s3') || method_exists( $as3cf, 'is_attachment_served_by_provider' ) ) { // ben ben if ( method_exists( $as3cf, 'is_attachment_served_by_provider' ) ) {
    // ben ben $s3_object = $as3cf->is_attachment_served_by_provider( $attachment_id, true );
    if ( method_exists( $as3cf, 'is_attachment_served_by_s3' ) ) {
    $s3_object = $as3cf->is_attachment_served_by_s3( $attachment_id, true );
    } else {
    $s3_object = $as3cf->is_attachment_served_by_provider( $attachment_id, true );
    }
    $size_prefix = dirname( $s3_object['key'] );
    $size_file_prefix = ( '.' === $size_prefix ) ? '' : $size_prefix . '/';
    if ( ! empty( $size_details ) && is_array( $size_details ) ) {
    $s3_object['key'] = path_join( $size_file_prefix, $size_details['file'] );
    } elseif ( ! empty( $uf_file_path ) ) {
    //Get the File path using basename for given attachment path
    $s3_object['key'] = path_join( $size_file_prefix, wp_basename( $uf_file_path ) );
    }

    //Try to download the attachment
    if ( $s3_object && is_object( $as3cf->plugin_compat ) && method_exists( $as3cf->plugin_compat, 'copy_provider_file_to_server' ) ) {
    //Download file
    $file = $as3cf->plugin_compat->copy_provider_file_to_server( $s3_object, $uf_file_path );
    }

    if ( $file ) {
    return $file;
    }
    }

    //If we don't have the file, Try it the basic way
    if ( ! $file ) {
    $s3_url = $this->is_image_on_s3( $attachment_id );

    //If we couldn't get the image URL, return false
    if ( is_wp_error( $s3_url ) || empty( $s3_url ) || ! $s3_url ) {
    return false;
    }

    if ( ! empty( $size_details ) ) {
    //If size details are available, Update the URL to get the image for the specified size
    $s3_url = str_replace( wp_basename( $s3_url ), $size_details['file'], $s3_url );
    } elseif ( ! empty( $uf_file_path ) ) {
    //Get the File path using basename for given attachment path
    $s3_url = str_replace( wp_basename( $s3_url ), wp_basename( $uf_file_path ), $s3_url );
    }

    //Download the file
    $temp_file = download_url( $s3_url );
    if ( ! is_wp_error( $temp_file ) ) {
    $renamed = @copy( $temp_file, $uf_file_path );
    unlink( $temp_file );
    }

    //If we were able to successfully rename the file, return file path
    if ( $renamed ) {

    return $uf_file_path;
    }
    }

    return false;
    }

    /**
    * Check if file exists for the given path
    *
    * @param string $attachment_id
    * @param string $file_path
    *
    * @return bool
    */
    function does_image_exists( $attachment_id = '', $file_path = '' ) {
    global $as3cf;
    if ( empty( $attachment_id ) || empty( $file_path ) ) {
    return false;
    }
    //Return if method doesn't exists
    if ( ! method_exists( $as3cf, 'is_attachment_served_by_s3') && ! method_exists( $as3cf, 'is_attachment_served_by_provider' ) ) { //ben ben if ( ! method_exists( $as3cf, 'is_attachment_served_by_provider' ) ) {
    error_log( "Couldn't find method is_attachment_served_by_provider or is_attachment_served_by_s3." ); // ben ben

    return false;
    }
    //Get s3 object for the file
    // ben ben $s3_object = $as3cf->is_attachment_served_by_provider( $attachment_id, true );
    if ( method_exists( $as3cf, 'is_attachment_served_by_s3' ) ) {
    $s3_object = $as3cf->is_attachment_served_by_s3( $attachment_id, true );
    } else {
    $s3_object = $as3cf->is_attachment_served_by_provider( $attachment_id, true );
    }

    $size_prefix = dirname( $s3_object['key'] );
    $size_file_prefix = ( '.' === $size_prefix ) ? '' : $size_prefix . '/';

    //Get the File path using basename for given attachment path
    $s3_object['key'] = path_join( $size_file_prefix, wp_basename( $file_path ) );

    //Get bucket details
    $bucket = $as3cf->get_setting( 'bucket' );
    $region = $as3cf->get_setting( 'region' );

    if ( is_wp_error( $region ) ) {
    return false;
    }

    // ben ben $s3client = $as3cf->get_provider_client( $region );
    if ( method_exists( $as3cf, 'get_s3client' ) ) {
    $s3client = $as3cf->get_s3client( $region );
    } else {
    $s3client = $as3cf->get_provider_client( $region );
    }

    // If we still have the older version of S3 Offload, use old method.
    if ( method_exists( $s3client, 'doesObjectExist' ) ) {
    $file_exists = $s3client->doesObjectExist( $bucket, $s3_object['key'] );
    } else {
    $file_exists = $s3client->does_object_exist( $bucket, $s3_object['key'] );
    }

    return $file_exists;
    }

    /**
    * Check if the file is served by S3 and download the file for given path
    *
    * @param string $file_path Full file path
    * @param string $attachment_id
    * @param array $size_details Array of width and height for the image
    *
    * @return bool|string False/ File Path
    */
    function maybe_download_file( $file_path = '', $attachment_id = '', $size_details = array() ) {
    if ( empty( $file_path ) || empty( $attachment_id ) ) {
    return false;
    }
    //Download if file not exists and served by S3
    if ( ! file_exists( $file_path ) && $this->is_image_on_s3( $attachment_id ) ) {
    return $this->download_file( $attachment_id, $size_details, $file_path );
    }

    return false;
    }

    /**
    * Checks if we've backup on S3 for the given attachment id and backup path
    *
    * @param string $attachment_id
    * @param string $backup_path
    *
    * @return bool
    */
    function backup_exists_on_s3( $exists, $attachment_id = '', $backup_path = '' ) {
    //If the file is on S3, Check if backup image object exists
    if ( $this->is_image_on_s3( $attachment_id ) ) {
    return $this->does_image_exists( $attachment_id, $backup_path );
    }

    return $exists;
    }

    /**
    * Update setting status - disable it if Gutenberg is not active.
    *
    * @since 2.8.1
    *
    * @param bool $disabled Setting status.
    *
    * @return bool
    */
    public function setting_status( $disabled ) {
    if ( ! class_exists( 'Amazon_S3_And_CloudFront' ) && ! class_exists( 'Amazon_S3_And_CloudFront_Pro' ) ) {
    $disabled = true;
    }

    return $disabled;
    }

    }

    global $wpsmush_s3;
    $wpsmush_s3 = new WpSmushS3();

    }

    if ( class_exists( 'AS3CF_Plugin_Compatibility' ) && ! class_exists( 'wp_smush_s3_compat' ) ) {
    class wp_smush_s3_compat extends AS3CF_Plugin_Compatibility {

    function __construct() {
    $this->init();
    }

    function init() {
    //Plugin Compatibility with Amazon S3
    add_filter( 'as3cf_get_attached_file', array( $this, 'smush_download_file' ), 11, 4 );
    }

    /**
    * Download the attached file from S3 to local server
    *
    * @param $url
    * @param $file
    * @param $attachment_id
    * @param $s3_object
    */
    function smush_download_file( $url, $file, $attachment_id, $s3_object ) {

    global $as3cf, $wpsmush_settings, $wp_smush;

    //Return if integration is disabled, or not a pro user
    if ( ! $wpsmush_settings->settings['s3'] || ! $wp_smush->validate_install() ) {
    return $url;
    }

    //If we already have the local file at specified path
    if ( file_exists( $file ) ) {
    return $url;
    }

    //Download image for Manual and Bulk Smush
    $action = ! empty( $_GET['action'] ) ? $_GET['action'] : '';
    if ( empty( $action ) || ! in_array( $action, array( 'wp_smushit_manual', 'wp_smushit_bulk' ) ) ) {
    return $url;
    }

    //If the plugin compat object is not available, or the method has been updated
    if ( ! is_object( $as3cf->plugin_compat ) || ! method_exists( $as3cf->plugin_compat, 'copy_image_to_server_on_action' ) ) {
    return $url;
    }

    $as3cf->plugin_compat->copy_image_to_server_on_action( $action, true, $url, $file, $s3_object );
    }

    }

    global $wpsmush_s3_compat;
    $wpsmush_s3_compat = new wp_smush_s3_compat();
    }

  • Predrag Dubajic
    • Support

    Hi Ben,

    The latest Smush version has only that patch for the S3 Offload integration, so if you’re using an older version of S3 Offload you can install the previous version of Smush as well to get them working together.

    Our developers are working on a new version of Smush with some other improvements and fixes and I we have passed this ticket over to them so they can check it out further.

    Best regards,

    Predrag

  • Ben
    • The Incredible Smush

    >> The latest Smush version has only that patch for the S3 Offload integration

    The patch breaks S3 for most users as I have shown. My “patch” for your “patch” makes it work for all versions of the s3 plugin…you guys should publish this asap

  • Predrag Dubajic
    • Support

    Hi Ben,

    Devs will review this for one of the next releases, but if you’re using updated Smush version you should really update S3 Offload as well, if you keep working on the older S3 version than you can keep older Smush installed.

    This is not a common use case because there’s rarely someone that would update one plugin but leave rest of them outdated.

    Just to be clear, I’m not saying that this will be ignored, just that it’s not something critical at this moment.

    Best regards,

    Predrag

  • Ben
    • The Incredible Smush

    Predrag Dubajic

    As I mentioned earlier…most people haven’t and don’t want to upgrade to version 2.0 people are reporting issues. So most people aren’t and the stats on wordpress.org show that

    Regardless of the stats, my solution has zero downside and is a more responsible way to code things.

    Its just irresponsible bad coding not to use my solution…seriously…shaking my head. I handed the solution on a silver plate, all you had to do is use it. The fixes are straight forward. Its annoying to have to go through code and fix mistakes on releases…especially the times I give the solution. If the developer actually reviews this thread he will see that, so PLEASE forward this to him so he see’s it and can update his code asap.

    Lets get it together guys, all on the same team here to make better products I hope.

  • Predrag Dubajic
    • Support

    Hi Ben,

    I’m sorry if my previous message was unclear, I meant in no way that this is not being looked into, we have passed it over to our developers straight away so they can review it for one of the future releases.

    However, new version of Smush was already deep into development and was sent to QA for testing so this didn’t make into that, but we have new version already being developed and this is on the to-do list there.

    We appreciate you providing us with the possible solution but it’s not something that we can just copy/paste in the plugin and release it, it needs to be reviewed and approved by the developers, and then tested by the QA before it can be pushed live.

    Hope that you understand and that this explains better what’s going on with this report.

    Best regards,

    Predrag

  • Ben
    • The Incredible Smush

    i wrote a long response, but I’m just giving up. programming and QA is difficult, but they should have seen this issue with proper testing, and especially with me pointed it out.

    If you are receiving a response from your QA that the development cycle not able to apply changes to code that breaks people’s system is the reason why this isn’t fixed…then that’s…i don’t know what to say about it. So I’m gonna give up. You guys were warned about that bad code, I shouldn’t be the guy fixing code your QA should be. Find better QA. Good luck.

  • Predrag Dubajic
    • Support

    Hi Ben,

    I understand your frustration but do note that we have a number of plugins, each one with its list of tasks that are being handled by our devs on a daily basis, and we need to put a priority on issues that are affecting our members the most.

    We had just a few reports about S3 Offload and Smush and everyone else has updated S3 plugin which resolved the issue so this fix didn’t get high on the priority list.

    As I mentioned, that doesn’t mean that we have given up on this or ignored it, just that other things had higher priority and new version was already under testing and was getting ready for a release, and devs will look into this for next release.

    Best regards,

    Predrag

  • Anton Vanyukov
    • Ex Staff

    Hi Ben,

    I have read over this thread. I also debugged your issue. I’ll try to answer all your questions.

    1. Backward compatibility issue with WP Offload S3.

    Thank you for the proposed fix. I have modified it to be more universal and it will be available with the next release. I understand that from your side it looks like the fix is a simple one and should have been released last week. But you must understand how QA works: once the release is prepared, it is thoroughly checked and tested against all possible scenarios, errors and bugs. This takes time. And to get a proper release out, it will either take time to test and verify, or it can lead to issues like this one with the backward compatibility errors. And you are implying that QA needs to properly test things but at the same time you are requesting that your proposed fix is merged ASAP. I hope that you understand that both things can’t happen at once – either it’s a stable release that takes time to properly test, or it’s a quick fix that can lead to issues.

    2. get_attached_file()

    The Smush get_attached_file() function is a wrapper function for the WordPress get_attached_file() method and should return either the local file path (if the Remove Files From Server option is not set in the WP Offload S3 plugin) or the original S3 path with the ‘s3’ string, not the CNAME URL. This happens, because of the get_attached_file filter that is triggering the get_stream_wrapper_file method in the WP Offload S3 plugin, which in turn will call the get_stream_wrapper_protocol() with a predefined ‘s3’ protocol:

    protected function get_stream_wrapper_protocol( $region ) {
    $protocol = 's3';
    $protocol .= str_replace( '-', '', $region );

    return $protocol;
    }

    I believe you might have some specific configuration that is causing an error. I have set up my test environment according to the WP Offload Media instructions:

    https://deliciousbrains.com/wp-offload-media/doc/cloudfront-setup/

    https://deliciousbrains.com/wp-offload-media/doc/custom-domain-https-cloudfront/

    With this setup I am not getting any errors. If you have a different configuration, please explain how I can reproduce it.

    Best regards,

    Anton

  • Ben
    • The Incredible Smush

    Anton Vanyukov

    please go into this file: lib/class-wp-smush-resize.php

    go to where the function is_animated is defined.

    in the first line of that file either email yourself the variable $file_path or send it to your log file.

    Network setup, on subblog other than blog 1. PHP 7.2 latest version. Subfolder install. Make sure the options of “resize image” is turned on AND then perform 2 more tests:

    1. Smush on load

    2. Smush not on load

    Then visit your directory (networking.com/subblog2/wp-admin/upload.php)

    Report what you see with that variable in the 2 different loads, and then also when visiting your media directory on wordpress with list view enabled. Tell me what you see in those scar

    IF it is a local file and you are not seeing any error…then you probably have the setting in your S3 plugin settings to keep the local file. So your set up keep duplicates. I am trying to build a scalable system on less expensive servers…so i have “remove local file” as would be the common use case.

    So i my code to fix things I’m using s3 functions used for both version 2.0 and 1.2.3 to check the existance and then the path and sending it to that function as the fix.

    Without the fix, I am seeing that function try to test the full local path to the file and getting the fail. As a result because of the various fails…when customers have imported layous in divi, the images were supposed to process through smush, the error happens because there is not try/fail error checking in either divi or this plugin, and then the import fails or deletes the data somehow and then the customer not knowing whats going on clicks save and then all the data they had gets written over with blank data…and then their site page is effectively deleted. The front end user work around is the turn off autoresize and stopping auto smush on load. But most people want those amazing features so that’s not a great solution.

    It all tracks back to this.

    SO regarding QA…its a pretty significant issue. And if 2 releases occur after i have given the issue to you guys, and the QA cycle can’t adapt to this significant issue…then we gotta QA..the QA system. We both have the same goals, lets really review this scenario of a customer pointing out an obvious issue and then the development cycle you guys employ not being able to adapt fast enough. AND having no communication with customers that this issue is existing. We could definitely do better on all these fronts and deliver some really great software.

    • Anton Vanyukov
      • Ex Staff

      Hi Ben,

      Sorry for taking some time to debug this. But I was finally able to replicate. I’ve refactored the code and moved the is_animated check to the actual Smushing process, when all the image are 100% available. The animated status for GIF images will be stored in a meta table for faster access later on. This should also speed up the media library. The change will make it to the next release.

      Also, if you’re interested, it should be possible to download the required file locally from S3 to do all the manipulations with this code:

      // Get the file path for backup.
      $attachment_file_path = WP_Smush_Helper::get_attached_file( $attachment_id );

      // Download if not exists.
      do_action( 'smush_file_exists', $attachment_file_path, $attachment_id );

      Thanks for taking the time to point this out.

      Best regards,

      Anton

  • Ben
    • The Incredible Smush

    The animated status for GIF images will be stored in a meta table for faster access later on.

    Glad we are on the same page on how to handle that now.

    Regarding your “smushing process”, I’ve only debugged the areas creating fatal crashes.

    I was under the assumption that a re-smushing or smushing from items stored in S3 would send http url paths to your s3 api directly and then the api would spit them back to me locally and then upload them to s3 to replace the existing items.

    Are you saying that is not happening? I’m confused where you are suggesting this code you are suggesting should be used. Thanks Anton Vanyukov

    Also Anton Vanyukov please confirm seeing the very simple function checks needed for the two different function names based on version for s3 needed that i showed you guys. I want to be able to “upgrade” without worries about manipulating your code.

    Thanks.

    • Anton Vanyukov
      • Ex Staff

      Ben,

      I was under the assumption that a re-smushing or smushing from items stored in S3 would send http url paths to your s3 api directly and then the api would spit them back to me locally and then upload them to s3 to replace the existing items.

      Image is downloaded locally, all the optimization is performed on the image, then it is uploaded back. The code was just for reference.

      Yes, I confirm that both versions of the S3 plugin will be supported in the upcoming release.

      Best regards.

      Anton

  • Ben
    • The Incredible Smush

    Anton Vanyukov

    Cool. Final question, in regards to this function:

    function get_attached_file( $attachment_id ) {
    if ( empty( $attachment_id ) ) {
    return false;
    }

    $file_path = get_attached_file( $attachment_id );
    if ( ! empty( $file_path ) && strpos( $file_path, 's3' ) !== false ) {
    $file_path = get_attached_file( $attachment_id, true );
    }

    return $file_path;
    }

    Not understanding the conditional logic of:

    strpos( $file_path, 's3' ) !== false

    Is this logic right for user with a s3 c-name setup and the s3 set up of removing local files? I think there are updated functions specifically set up for this check to be used. You probably have now seen this, but just want to re-confirm so that there are no mistakes.

    • Anton Vanyukov
      • Ex Staff

      Ben,

      The files that are stored on S3 will always have the ‘s3’ string present in the URL. Think of it as a direct path for the file. The CNAME is like an alias. You can reach the file both ways. But the CNAME is used for users and… maybe CEO, the original path with ‘s3’ is used for all the internal stuff. That is how the WP Offload Media plugin handles the files.

      Best regards,

      Anton

  • Ben
    • The Incredible Smush

    Anton Vanyukov

    To be honest i don’t quite understand the recursive operations of get_attached_file in class WpSmushHelper

    But I don’t things are operating as you expect them, but maybe it wont matter after you refactor things. Please do a simple test to the class WpSmushHelper function get_attached_file by inserting the following statments I have commented…i am just emailing them, but you can send them to your logs:

    /**
    * Return unfiltered file path
    *
    * <a title=@param href=/profile/param>param</a> $attachment_id
    *
    * @return bool
    */
    function get_attached_file( $attachment_id ) {
    if ( empty( $attachment_id ) ) {
    return false;
    }

    $file_path = get_attached_file( $attachment_id );
    // ben ben
    mail('YOUREMAIL@gmail.com', 'PRE-TEST 1', $file_path);
    if ( ! empty( $file_path ) && strpos( $file_path, 's3' ) !== false ) {
    $file_path = get_attached_file( $attachment_id, true );
    mail('YOUREMAIL@gmail.com', 'PRE-TEST 2', $file_path);
    }

    return $file_path;
    }

    ‘PRE-TEST 1’, $file_path will show

    s3uswest1://files.mydomain.com/content/uploads/sites/105/2018/03/02014405/fashion-03.png

    ‘PRE-TEST 2’, $file_path will show

    /srv/users/serverpilot/apps/wordpress/public/wp-content/uploads/sites/105/2018/06/cosmetic-shop-icon-bag.png

    files.mydomain.com is my c-name

    So its picking up the s3 match from the bucket server location, and then of course its returning the full non s3 path.

    Just want to make sure you are aware of that.